luxolus
/
open-consul
2
0
Fork 0
Code Issues 1 Pull Requests Packages Releases Wiki Activity

nia/docs 0.5.0 (#12381) 

* docs/nia: new configuration for services condition & source_input (#11646)

* docs/nia: new configuration for services condition

* docs/nia: new configuration for services source_input

* reword filter and cts_user_defined_meta

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

* Update service block config to table format

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

* Remove deprecated driver.working_dir (#11831)

* Deprecate workspace_prefix for now workspaces.prefix (#11836)

* docs/nia: new config field names for services condition/source_input (#11896)

* docs/nia: new config field `names` for services condition/source_input

* Remove language about 'default condition' and services condition relation to services list

Context:
 - Added a new `names` field to condition/source_input "services"
 - `names` or `regexp` must be configured for condition/source_input "services"

This therefore:
 - Removed relationship between condition/source_input "services" and
 task.services list
 - Removed concept of "default condition" i.e. condition "services" must be
 configured with `names` or `regexp`, there is no meaningful unconfigured default

Change: remove language regarding "default condition" and relationship with services list

* docs/nia: Update paramters to table format

Changes from a bulleted list to a table. Also adds the possible response codes
and fixes the update example response to include the inspect object.

* docs/nia: Delete task API and CLI

* docs/nia: Update wording for run values

Co-authored-by: Michael Wilkerson <62034708+wilkermichael@users.noreply.github.com>

* docs/nia: require condition "catalog-services" block's regexp to be configured (#11915)

Changes:
 - Update Catalog Services Condition configuration docs to new table format
 - Rewrite `regexp` field docs to be required, no longer optional
 - Remove details about `regexp` field's original default behavior when the
 field was optional

* docs/nia: Update status API docs to table format

* Cleaner wording for response descriptions

Co-authored-by: mrspanishviking <kcardenas@hashicorp.com>

* docs/nia - 'source_includes_var' changes (#11939)

* docs/nia - condition "services" new field source_includes_var

 - Add new configuration details for condition "services" block's
 `source_includes_var` field.
 - Note: this field's description is worded differently from condition type's
 `source_includes_var` since a services variable is always required (unlike
 other vars) for CTS modules.
 - Also worded in a way to anticipate renaming to `use_as_module_input`

* docs/nia - change 'source_includes_var' default value from false to true

 - Update configs
 - Table-ify Consul-KV condition (reuse wording from Consul-KV source input)

* docs/nia - reword task execution page for source_includes_var changes

 - Note: switched to using "module input" language over "source input" language.
 Separate PR will make a mass change across docs
 - Slim down general task condition section to have fewer details on module input
 - Updated services, catalog-services, and consul-kv condition sections for
 source_includes_var
 - Add config page links for details

* Improve CTS acronym usage
- Use Consul-Terraform-Sync at the first instance with CTS in brackets - Consul-Terraform-Sync (CTS) and then CTS for all following instances on a per-page basis.
- some exceptions: left usage of the term `Consul-Terraform-Sync` in config examples and where it made sense for hyperlinking

* Improve CTS acronym usage (part 2) (#11991)

Per page:
- At first instance in text, use "Consul-Terraform-Sync (CTS)"
- Subsequent instances in text, use "CTS"

* Update schedule condition config to table format

* Update config tables with type column

* docs/nia: Update required fields values

Standardizing Required/Optional over boolean values.

* docs/nia: Standardize order of columns

Updated Required to come before Type, which is how the configurations are formatted. Also
changed the empty strings to "none" for default values.

* Deprecate port CLI option for CTS and updated example usage

* docs/nia cts multiple source input configuration updates (#12158)

* docs/nia cts multiple source input configuration updates

CTS expanded its usage of `source_input` block configurations and added
some restrictions. This change accounts for the following changes:

- `source_input` block can be configured for a task. No longer restricting to
scheduled task
- Multiple `source_input` blocks can be configured for a task. No longer
restricting to one
- Task cannot have multiple configurations defining the same variable type

Future work: We're planning to do some renaming from "source" to "module" for
v0.5. These changes are made in the code and not yet in the docs. These will be
taken care of across our docs in a separate PR. Perpetuating "source" in this
PR to reduce confusion.

* Apply suggestions from code review

Co-authored-by: mrspanishviking <kcardenas@hashicorp.com>

* Apply suggestions from code review

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

* code review feedback

Co-authored-by: mrspanishviking <kcardenas@hashicorp.com>
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

* Add "Consul object" glossary entry

Changes:
 - Add "Consul object" to CTS glossary
 - Format glossary terms so that they can be linked
 - Add link to "Consul object" glossary entry

* Reorganize source_input limitations section

Co-authored-by: findkim <6362111+findkim@users.noreply.github.com>

Co-authored-by: mrspanishviking <kcardenas@hashicorp.com>
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
Co-authored-by: findkim <6362111+findkim@users.noreply.github.com>

* docs/nia: overview of config streamlining deprecations (#12193)

* docs/nia: overview of config streamlining deprecations

* Update config snippets to use CodeTabs

* Apply code review feedback suggestions

Co-authored-by: mrspanishviking <kcardenas@hashicorp.com>

* Apply suggestions from code review

Co-authored-by: mrspanishviking <kcardenas@hashicorp.com>

* Clarify source table language

* Add use_as_module_input callout

Co-authored-by: mrspanishviking <kcardenas@hashicorp.com>

* docs/nia: deprecate "services" field and "service" block (#12234)

* Deprecate `services` field

Did a search on "`services`", "`task.services`", "services list", and "services
field"

Changes:
 - In config docs, mark `services` field as deprecated and `condition` block
 as required.
 - For necessary references to `services` field, mark with "(deprecated)" e.g.
 when listing all options for source input
 - Remove unnecessary references to `services` field from docs e.g. any docs
 encouraging use of `services`
 - Replace `services` field with `condition` / `module_input` "services" in
 config snippets and explanations

* Deprecate `service` block

Did a search for "service block", "`service`", and "service {"

Changes:
 - In config docs, mark `service` block as deprecated
 - For necessary references to `service` block, mark with "(deprecated)"
 - Remove unnecessary references to `service` block from docs

* Fix service block typos in config snippet

service block is singular and not plural

* docs/nia: deprecate "source includes var" and "source input" (#12244)

* Deprecate `source_includes_var` field

Did a search for "source_includes_var" and an audit of "include"

Changes
 - In config docs, mark `source_includes_var` field as deprecated
 - In config docs, add new field for `use_as_module_input`
 - For necessary references to `source_includes_var`, mark with "(deprecated)"
 - Audit and update "include" language

* Deprecate `source_input` field and language

Did a search and replace for "source_input", "source-input", "source input"

Changes:
 - In config docs, mark `source_input` field as deprecated
 - In config docs, add new entry for `module_input`
 - For necessary references to `source_input`, mark with "(deprecated)"
 - Remove or replace "source*input" with "module*input"

Note: added an anchor link alias e.g. `# Module Input ((#source-input))` for
headers that were renamed from "Source Input" so that bookmarked links won't
break

* Update config streamlining release removal version to 0.8

* remove duplicate bullet

* docs/nia: deprecate `source` (#12245)

* Update "source" field in config snippets to "module"

* Deprecate task config `source` field

Did a search and replace for "source" and "src"

Changes:
 - In config docs, mark `source` field as deprecated
 - In config docs, add new entry for `module`
 - Remove or replace "source" with "module"

* Deprecate Status API Event `source` field

Changes:
 - Mark `source` field as deprecated
 - Add new entry for `module`

* docs/nia - Get Task API docs & Task Status API deprecations (#12303)

* docs/nia - Get Task API

Added a Task Object section intended to be shared with the Create Task API

* docs/nia - Deprecate non-status fields from Task Status API

Deprecate the fields that Get Task API replaces

* docs/nia - Align API docs on `:task_name` request resource

Followed a convention found in Nomad docs

* docs/nia - misc fixes

Context for some:
 - remove "" from license_path for consistency - do not specify the default
 value when empty string
 - remove "optional" language from task condition. we want to move towards it
 being required

* docs/nia - add new columns to API Task Object

* Added Create Task API documentation

* Added create task CLI documentation

* addressed code review comments

* fixed example

* docs/nia: Update task delete with async behavior

CTS delete task command is now asynchronous, so updating docs to reflect
this new behavior.

* update create task CLI with new changes from code

* update create task api and cli
- update curl command to include the json header
- update example task names to use 'task_a' to conform with other examples

* docs/nia: Fix hyphens in CTS CLI output

* docs/nia: Add auto-approve option in CLI

* docs/nia: Clarify infrastructure is not destroyed on task deletion

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
Co-authored-by: Kim Ngo <6362111+findkim@users.noreply.github.com>
Co-authored-by: Melissa Kam <mkam@hashicorp.com>
Co-authored-by: Melissa Kam <3768460+mkam@users.noreply.github.com>
Co-authored-by: Michael Wilkerson <62034708+wilkermichael@users.noreply.github.com>
Co-authored-by: mrspanishviking <kcardenas@hashicorp.com>
Co-authored-by: Michael Wilkerson <mwilkerson@hashicorp.com>
Co-authored-by: AJ Jwair <aj.jwair@hashicorp.com>
This commit is contained in:
lornasong 2022-02-23 14:22:34 -05:00 committed by GitHub
parent 091f7670c8
commit 4bc423204a
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
21 changed files with 1344 additions and 441 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
website/content/docs/nia/api/index.mdx
View File

@ -7,7 +7,7 @@ description: >-
# Consul-Terraform-Sync API
When running in [daemon mode](/docs/nia/cli#daemon-mode), Consul-Terraform-Sync serves an HTTP API interface. Details of the available API endpoints is available in the navigation to the left.
When running in [daemon mode](/docs/nia/cli#daemon-mode), Consul-Terraform-Sync (CTS) serves an HTTP API interface. Details of the available API endpoints is available in the navigation to the left.
### Port

91
website/content/docs/nia/api/status.mdx
View File

@ -8,7 +8,7 @@ description: >-
The `/status` endpoints share status-related information for tasks. This information is available for understanding the status of individual tasks and across tasks.
The health status value is determined by aggregating the success or failure of the event of a task detecting changes in Consul services and then updating network infrastructure. Currently, only the 5 most recent events are stored in Consul-Terraform-Sync. For more information on the hierarchy of status information and how it is collected, see [Status Information](/docs/nia/tasks#status-information).
The health status value is determined by aggregating the success or failure of the event of a task detecting changes in Consul services and then updating network infrastructure. Currently, only the 5 most recent events are stored in Consul-Terraform-Sync (CTS). For more information on the hierarchy of status information and how it is collected, see [Status Information](/docs/nia/tasks#status-information).
## Overall Status
@ -21,18 +21,24 @@ This endpoint currently returns the overall status information for all tasks.
#### Request Parameters
Currently no request parameters are offered for the overall status API.
#### Response Statuses
| Status | Reason |
| ------ | ---------------- |
| 200 | Successfully retrieved the status |
#### Response Fields
* `task_summary` - Summary of task information. See [Task Status API](/docs/nia/api#task-status) to learn more about how health status is determined.
* `status` - Summary count of how many tasks have each health status value
* `successful` - (int) The number of tasks that have a 'successful' health status
* `errored` - (int) The number of tasks that have a 'errored' health status
* `critical` - (int) The number of tasks that have a 'critical' health status
* `unknown` - (int) The number of tasks that have an 'unknown' health status
* `enabled` - Summary count of how many tasks are enabled or disabled
* `true` - (int) The number of tasks that are enabled
* `false` - (int) The number of tasks that are disabled
| Name | Type | Description |
| ----- | ------ | ------------------ |
|`status` | object | The summary count of how many tasks have a given health status value
|`status.successful` | integer | The number of tasks that have a `successful` health status.
|`status.errored` | integer | The number of tasks that have an `errored` health status.
|`status.critical` | integer | The number of tasks that have a `critical` health status.
|`status.unknown` | integer | The number of tasks that have an `unknown` health status.
|`enabled` | object | The summary count of how many tasks are enabled or disabled.
|`enabled.true` | integer | The number of tasks that are enabled.
|`enabled.false` | integer | The number of tasks that are disabled.
#### Example
@ -71,41 +77,54 @@ Task health status value is determined by the success or failure of all stored [
| Method | Path | Produces |
| ------ | ------------------- | ------------------ |
| `GET` | `/status/tasks/:task` | `application/json` |
| `GET` | `/status/tasks/:task_name` | `application/json` |
#### Request Parameters
* `task` - (string) Option to specify the name of the task to return in the response. If not specified, all tasks are returned in the response.
* `include` - (string) Only accepts the value "events". Use to include stored event information in response.
* `status` - (string) Only accepts health status values "successful", "errored", "critical", or "unknown". Use to filter response by tasks that have the specified health status value. Recommend setting this parameter when requesting all tasks i.e. no `task` parameter is set.
| Name | Required | Type | Description | Default |
| -------- | -------- | ------ | ------------------ | ------- |
|`:task_name` | Optional | string | Option to specify the name of the task to return in the response. If not specified, all tasks are returned in the response.| none
|`include` | Optional | string | Only accepts the value `"events"`. Use to include stored event information in response. | none
|`status` | Optional | string | Only accepts health status values `"successful"`, `"errored"`, `"critical"`, or `"unknown"`. Use to filter response by tasks that have the specified health status value. Recommend setting this parameter when requesting all tasks i.e. no `task` parameter is set. | none
#### Response Statuses
| Status | Reason |
| ------ | ---------------- |
| 200 | Successfully retrieved the task status |
| 404 | Task with the given name not found |
#### Response Fields
The response is a JSON map of task name to a status information structure with the following fields.
* `task_name` - (string) Name that task is configured with in Consul-Terraform-Sync.
* `status` - (string) Values are "successful", "errored", "critical", or "unknown". This is determined by the success or failure of all stored events on the network infrastructure update process for the task, as described earlier.
* `enabled` - (bool) Whether the task is enabled or not.
* `services` - (list[string]) List of the services configured for the task.
* `providers` - (list[string]) List of the providers configured for the task.
* `events_url` - (string) Relative URL to retrieve the event data stored for the task.
* `events` - [(list[Event])](/docs/nia/api#event) - List of stored events that inform the task's status. See section below for information on event data. This field is only included in the response upon request by setting the `?include=events` parameter. The relative URL for the request to include events can be retrieved from the `events_url` field.
| Name | Type | Description |
| ----------- | ------ | ------------------ |
|`task_name` | string | Name of the task as configured in CTS.
|`status` | string | Values are `"successful"`, `"errored"`, `"critical"`, or `"unknown"`. This is determined by the success or failure of all stored events on the network infrastructure update process for the task, as described earlier.
|`enabled` | boolean | Whether the task is enabled or not.
|`services` | list[string] | **Deprecated in CTS 0.5.0 and will be removed in a future major release.** List of the services configured for the task.
|`providers` | list[string] | **Deprecated in CTS 0.5.0 and will be removed in v0.8.0.** List of the providers configured for the task.
|`events_url` | string | Relative URL to retrieve the event data stored for the task.
|`events` | list[[Event](/docs/nia/api/status#event)] | List of stored events that inform the task's status. See [section below](/docs/nia/api/status#event) for information on event data. This field is only included in the response upon request by setting the `?include=events` parameter. The relative URL for the request to include events can be retrieved from the `events_url` field.
##### Event
Event represents the process of updating network infrastructure of a task. The data is captured in a JSON structure. For more details on the scope of an event, see [Event](/docs/nia/tasks#event).
* `id` - (string) UUID to uniquely identify the event.
* `success` - (bool) Indication of whether the event was successful or not.
* `start_time` - (time) Time when the event started.
* `end_time` - (time) Time when the event ended.
* `task_name` - (string) Name that task is configured with in Consul-Terraform-Sync.
* `error` Information when the event fails. Null when successful.
* `message` - (string) Error message that is returned on failure.
* `config`
* `services` - (list[string]) List of the services configured for the task.
* `source` - (string) Source configured for the task.
* `providers` - (list[string]) List of the providers configured for the task.
| Name | Type | Description |
| ----------- | ------ | ------------------ |
|`id` | string | UUID to uniquely identify the event.
|`success` | boolean | Indication of whether the event was successful or not.
|`start_time` | time | Time when the event started.
|`end_time` | time | Time when the event ended.
|`task_name` | string | Name that task is configured with in CTS.
|`error` | object | Information when the event fails. Null when successful.
|`error.message` | string | Error message that is returned on failure.
|`config` | object | **Deprecated in CTS 0.5.0 and will be removed in v0.8.0.** Configuration values for the task when it was run.
|`config.services` | list[string] | **Deprecated in CTS 0.5.0 and will be removed in v0.8.0.** List of the services configured for the task.
|`config.source` | string | **Deprecated in CTS 0.5.0 and will be removed in v0.8.0.** Module configured for the task.
|`config.providers` | list[string] | **Deprecated in CTS 0.5.0 and will be removed in v0.8.0.** List of the providers configured for the task.
#### Example: All Task Statuses
@ -180,9 +199,9 @@ Response:
"null"
],
"services": [
"web",
"web"
],
"source": "../modules/test_task"
"module": "../modules/test_task"
}
},
{
@ -199,7 +218,7 @@ Response:
"services": [
"web",
],
"source": "../modules/test_task"
"module": "../modules/test_task"
}
}
]

293
website/content/docs/nia/api/tasks.mdx
View File

@ -4,34 +4,224 @@ page_title: Consul-Terraform-Sync Tasks API
description: >-
Consul-Terraform-Sync Tasks API
---
# Tasks
The `/tasks` endpoints modify the tasks that Consul-Terraform-Sync is responsible for running.
The `/tasks` endpoints interact with the tasks that Consul-Terraform-Sync (CTS) is responsible for running.
## Get Task
This endpoint returns information about an existing task.
| Method | Path | Produces |
| ------ | ------------------- | ------------------ |
| `GET` | `/tasks/:task_name` | `application/json` |
#### Request Parameters
| Name | Required | Type | Description | Default |
| ------------ | -------- | ------ | ------------------------------------------ | ------- |
| `:task_name` | Required | string | Specifies the name of the task to retrieve | none |
#### Response Statuses
| Status | Reason |
| ------ | ---------------------------------------------------- |
| 200 | Successfully retrieved and returned task information |
| 404 | Task with the given name not found |
#### Response Fields
| Name | Type | Description |
| ------------ | ------ | ---------------------------------------------------------------------------------- |
| `request_id` | string | The ID of the request. Used for auditing and debugging purposes. |
| `task` | object | The task's configuration information. See [task object](#task-object) for details. |
#### Example: Retrieve a task
Request:
```shell-session
$ curl --request GET \
localhost:8558/v1/tasks/task_a
```
Response:
```json
{
"request_id": "b7559ab0-5111-381b-367a-0dfb7e216d41",
"task": {
"buffer_period": {
"enabled": true,
"max": "20s",
"min": "5s"
},
"condition": {
"consul_kv": {
"datacenter": "",
"namespace": "",
"path": "my_key",
"recurse": false,
"use_as_module_input": true
}
},
"description": "task triggering on consul-kv",
"enabled": true,
"module": "path/to/module",
"module_input": {
"services": {
"cts_user_defined_meta": {},
"datacenter": "",
"filter": "",
"names": ["api"],
"namespace": ""
}
},
"name": "task_a",
"providers": [],
"variables": {},
"version": ""
}
}
```
# Create Task
This endpoint allows for creation of a task. It only supports creation of a new task, and does not support updating a task.
| Method | Path | Produces |
| ------ | -------- | ------------------ |
| `POST` | `/tasks` | `application/json` |
#### Request Parameters
| Name | Required | Type | Description | Default |
| ----- | -------- | ------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- |
| `run` | Optional | string | Values can only be `"inspect"` and `"now"`.<br/><ul><li>`"inspect"`: Does not create the task but returns information on if/how resources would be changed for the proposed task creation.</li><li> `"now"`: Creates the task accordingly and immediately runs the task, rather than allowing the task to run at the natural daemon cadence</li></ul> | none |
#### Request Body
| Name | Required | Type | Description |
| ------ | -------- | ------ | ---------------------------------------------------------------------------------- |
| `task` | Required | object | The task's configuration information. See [task object](#task-object) for details. |
#### Response Statuses
| Status | Reason |
| ------ | ----------------------------- |
| 201 | Successfully created the task |
#### Response Fields
| Name | Type | Description |
| ------------ | ------ | ------------------------------------------------------------------------------------------------- |
| `request_id` | string | The ID of the request. Used for auditing and debugging purposes. |
| `task` | object | The task's configuration information after creation. See [task object](#task-object) for details. |
#### Example: Create a task
Payload:
```json
{
"task": {
"description": "Writes the service name, id, and IP address to a file",
"enabled": true,
"name": "task_a",
"providers": ["my-provider"],
"condition": {
"services": {
"names": ["web", "api"]
}
},
"module": "path/to/module"
}
}
```
Request:
```shell-session
$ curl --header "Content-Type: application/json" \
--request POST \
--data @payload.json \
localhost:8558/v1/tasks
```
Response:
```json
{
"request_id": "0428ed18-1359-874c-8b27-742e43ebd7e7",
"task": {
"buffer_period": {
"enabled": true,
"max": "25s",
"min": "5s"
},
"condition": {
"services": {
"cts_user_defined_meta": {},
"datacenter": "",
"filter": "",
"names": ["api", "web"],
"namespace": "",
"use_as_module_input": true
}
},
"description": "Writes the service name, id, and IP address to a file",
"enabled": true,
"module": "path/to/module",
"module_input": {},
"name": "task_a",
"providers": ["my-provider"],
"variables": {},
"version": ""
}
}
```
## Update Task
This endpoint allows patch updates to specifically supported fields for existing tasks. Currently only supports updating a task's [`enabled` value](/docs/nia/configuration#enabled-4).
| Method | Path | Produces |
| ------ | ------------------- | ------------------ |
| `PATCH` | `/tasks/:task_name` | `application/json` |
| Method | Path | Produces |
| ------- | ------------------- | ------------------ |
| `PATCH` | `/tasks/:task_name` | `application/json` |
#### Request Parameters
* `run` - (string) Values can be only be "inspect" and "now".
* "inspect": Does not update the task but returns information on if/how resources would be changed for the proposed task update.
* "now": Updates the task accordingly and immediately runs the task, rather than allowing the task to run at the natural daemon cadence
| Name | Required | Type | Description | Default |
| ------------ | -------- | ------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- |
| `:task_name` | Required | string | Specifies the name of the task to update | none |
| `run` | Optional | string | Values can only be `"inspect"` and `"now"`.<br/><ul><li>`"inspect"`: Does not update the task but returns information on if/how resources would be changed for the proposed task update.</li><li> `"now"`: Updates the task accordingly and immediately runs the task, rather than allowing the task to run at the natural daemon cadence</li></ul> | none |
#### Request Body
| Name | Required | Type | Description |
| --------- | -------- | ------- | -------------------------------------------------------- |
| `enabled` | Required | boolean | Whether the task is enabled to run and manage resources. |
#### Response Statuses
| Status | Reason |
| ------ | ---------------------------------- |
| 200 | Successfully updated the task |
| 404 | Task with the given name not found |
#### Response Fields
* `inspect` - Information on how resources would be changed given a proposed task update, similar to [inspect-mode](/docs/nia/cli#inspect-mode). This is only returned when passed the `run=inspect` request parameter
* `changes_present` - (bool) Whether or not changes were detected for running the proposed task update
* `plan` - (string) The Terraform plan generated for the proposed task update . Note: a non-empty string does not necessarily indicate changes were detected.
| Name | Type | Description |
| ------------------------- | ------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `inspect` | object | Information on how resources would be changed given a proposed task update, similar to [inspect-mode](/docs/nia/cli/#inspect-mode). This is only returned when passed the `run=inspect` request parameter. |
| `inspect.changes_present` | boolean | Whether or not changes were detected for running the proposed task update. |
| `inspect.plan` | string | The Terraform plan generated for the proposed task update. Note: a non-empty string does not necessarily indicate changes were detected. |
#### Example: Disable a task
Request:
```shell-session
$ curl --header "Content-Type: application/json" \
--request PATCH \
@ -40,6 +230,7 @@ $ curl --header "Content-Type: application/json" \
```
Response:
```json
{}
```
@ -47,6 +238,7 @@ Response:
#### Example: Inspect enabling a task
Request:
```shell-session
$ curl --header "Content-Type: application/json" \
--request PATCH \
@ -55,17 +247,88 @@ $ curl --header "Content-Type: application/json" \
```
Response if no changes present:
```json
{
"changes_present": false,
"plan": "No changes. Infrastructure is up-to-date. <truncated>"
"inspect": {
"changes_present": false,
"plan": "No changes. Infrastructure is up-to-date. <truncated>"
}
}
```
Response if changes present:
```json
{
"changes_present": true,
"plan": "<terraform plan truncated> Plan: 3 to add, 0 to change, 0 to destroy."
"inspect": {
"changes_present": true,
"plan": "<terraform plan truncated> Plan: 3 to add, 0 to change, 0 to destroy."
}
}
```
## Delete Task
This endpoint allows for deletion of existing tasks. It marks a task for deletion based on the name provided. If the task is not running, it will be deleted immediately. Otherwise, it will be deleted once the task is completed.
-> **Note:** Deleting a task will not destroy the infrastructure managed by the task.
| Method | Path | Produces |
| -------- | ------------------- | ------------------ |
| `DELETE` | `/tasks/:task_name` | `application/json` |
#### Request Parameters
| Name | Required | Type | Description | Default |
| ------------ | -------- | ------ | ------------------------------------------ | ------- |
| `:task_name` | Required | string | Specifies the name of the task to retrieve | none |
#### Response Statuses
| Status | Reason |
| ------ | ----------------------------------------- |
| 202 | Successfully marked the task for deletion |
| 404 | Task with the given name not found |
#### Response Fields
| Name | Type | Description |
| ------------ | ------ | ---------------------------------------------------------------- |
| `request_id` | string | The ID of the request. Used for auditing and debugging purposes. |
#### Sample Request
```shell-session
$ curl --request DELETE \
localhost:8558/v1/tasks/task_a
```
#### Sample Response
```json
{
"request_id": "9b23eea7-a435-2797-c71e-10c15766cd73"
}
```
## Task Object
The task object is used by the Task APIs as part of a request or response. It represents the task's [configuration information](/docs/nia/configuration#task).
| Name | Required | Type | Description | Default |
| ----------------------- | ------------ | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------ |
| `buffer_period` | object | Optional | The buffer period for triggering task execution. | The global buffer period configured for CTS. |
| `buffer_period.enabled` | bool | Optional | Whether the buffer period is enabled or disabled. | The global buffer period's enabled value configured for CTS. |
| `buffer_period.min` | string | Optional | The minimum period of time to wait after changes are detected before triggering the task. | The global buffer period's min value configured for CTS. |
| `buffer_period.max` | string | Optional | The maximum period of time to wait after changes are detected before triggering the task. | The global buffer period's max value configured for CTS. |
| `condition` | object | Required | The [condition](/docs/nia/configuration#task-condition) on which to trigger the task to execute. <br/><br/> If the task has the deprecated `services` field configured as a module input, it is represented here as `condition.services`. | none |
| `description` | string | Optional | The human readable text to describe the task. | none |
| `enabled` | bool | Optional | Whether the task is enabled or disabled from executing. | `true` |
| `module` | string | Required | The location of the Terraform module. | none |
| `module_input` | object | Optional | The additional [module input(s)](/docs/nia/configuration#task-source-input) that the tasks provides to the Terraform module on execution. <br/><br/> If the task has the deprecated `services` field configured as a module input, it is represented here as `module_input.services`. | none |
| `name` | string | Required | The unique name of the task. | none |
| `providers` | list[string] | Optional | The list of provider names that the task's module uses. | none |
| `variables` | map[string] | Optional | The map of variables that are provided to the task's module. | none |
| `version` | string | Optional | The version of the configured module that the task uses. | The latest version. |
| `terraform_version` | string | Optional | <EnterpriseAlert inline /> The version of Terraform to use for the Terraform Cloud workspace associated with the task. This is only available when used with the [Terraform Cloud driver](/docs/nia/configuration#terraform-driver). | The latest compatible version supported by the organization. |

10
website/content/docs/nia/architecture.mdx
View File

@ -7,8 +7,8 @@ description: >-
# Consul-Terraform-Sync Architecture
Consul-Terraform-Sync is a service-oriented tool for managing
network infrastructure near real-time. Consul-Terraform-Sync runs as a daemon
Consul-Terraform-Sync (CTS) is a service-oriented tool for managing
network infrastructure near real-time. CTS runs as a daemon
and integrates the network topology maintained by your Consul cluster with your
network infrastructure to dynamically secure and connect services.
@ -16,7 +16,7 @@ network infrastructure to dynamically secure and connect services.
[![Consul-Terraform-Sync Architecture](/img/nia-highlevel-diagram.svg)](/img/nia-highlevel-diagram.svg)
The diagram shows Consul-Terraform-Sync monitoring the Consul service catalog
The diagram shows CTS monitoring the Consul service catalog
for updates and utilizing Terraform to update the state of the infrastructure.
There are two principal aspects of Sync to know about corresponding to the
@ -27,7 +27,7 @@ update the infrastructure.
## Watcher and Views
Consul-Terraform-Sync monitors Consul for updates utilizing Consul's [Blocking
CTS monitors Consul for updates utilizing Consul's [Blocking
Queries](/api-docs/features/blocking) whenever supported, falling back on
polling when not. The watcher maintains a separate thread (known internally as
a view) for each value monitored, running any tasks that depend on that watched
@ -57,4 +57,4 @@ Each driver includes a set of providers that [enables support](/docs/nia/terrafo
The [Secure Consul-Terraform-Sync for Production](https://learn.hashicorp.com/tutorials/consul/consul-terraform-sync-secure?utm_source=WEBSITE&utm_medium=WEB_IO&utm_offer=ARTICLE_PAGE&utm_content=DOCS)
tutorial contains a checklist of best practices to secure your
Consul-Terraform-Sync installation for a production environment.
CTS installation for a production environment.

36
website/content/docs/nia/cli/index.mdx
View File

@ -7,11 +7,11 @@ description: >-
# Consul-Terraform-Sync Command (CLI)
Consul-Terraform-Sync is controlled via an easy to use command-line interface (CLI). Consul-Terraform-Sync is only a single command-line application: `consul-terraform-sync`. Consul-Terraform-Sync can be run as a daemon and execute CLI commands. When Consul-Terraform-Sync is run as a daemon, it acts as a server to the CLI commands. Users can use the commands to interact and modify the daemon as it is running. The complete list of commands is in the navigation to the left. Both the daemon and commands return a non-zero exit status on error.
Consul-Terraform-Sync (CTS) is controlled via an easy to use command-line interface (CLI). CTS is only a single command-line application: `consul-terraform-sync`. CTS can be run as a daemon and execute CLI commands. When CTS is run as a daemon, it acts as a server to the CLI commands. Users can use the commands to interact and modify the daemon as it is running. The complete list of commands is in the navigation to the left. Both the daemon and commands return a non-zero exit status on error.
## Daemon
When running as a daemon, there is no default configuration to run Consul-Terraform-Sync in a meaningful way. Setting a configuration flag `-config-file` or `-config-dir` is required. For example:
When running as a daemon, there is no default configuration to run CTS in a meaningful way. Setting a configuration flag `-config-file` or `-config-dir` is required. For example:
```shell-session
$ consul-terraform-sync -config-file=config.hcl
@ -21,13 +21,13 @@ To view a list of available flags, use the `-help` or `-h` flag.
### Modes
Consul-Terraform-Sync can be run as a daemon in different modes.
CTS can be run as a daemon in different modes.
#### Long-running Mode
Flag: none
Behavior: This is the default mode in which Consul-Terraform-Sync passes through a once-mode phase and then turns into a long running process. During the once-mode phase, the daemon will exit with a non-zero status if it encounters an error. After successfully passing through once-mode phase, it will begin a long-running process in which errors are logged and exiting is not expected behavior. Once beginning the long-running process, the daemon serves any API and command requests.
Behavior: This is the default mode in which CTS passes through a once-mode phase and then turns into a long running process. During the once-mode phase, the daemon will exit with a non-zero status if it encounters an error. After successfully passing through once-mode phase, it will begin a long-running process in which errors are logged and exiting is not expected behavior. Once beginning the long-running process, the daemon serves any API and command requests.
Usage: Intended to be run as a long running process after running once-mode successfully for given configuration and tasks.
@ -35,7 +35,7 @@ Usage: Intended to be run as a long running process after running once-mode succ
Flag: `-inspect`
Behavior: Consul-Terraform-Sync will display the proposed state changes for all tasks once and exit. No changes are applied in this mode. On encountering an error before completing, Consul-Terraform-Sync will exit with a non-zero status.
Behavior: CTS will display the proposed state changes for all tasks once and exit. No changes are applied in this mode. On encountering an error before completing, CTS will exit with a non-zero status.
Usage: Intended to be run before daemon-mode in order to confirm configuration is accurate and tasks would update network infrastructure as expected.
@ -51,20 +51,20 @@ Usage: Useful to debug one or more tasks to confirm configuration is accurate an
Flag: `-once`
Behavior: Consul-Terraform-Sync will run all tasks once with buffer periods disabled and exit. On encountering an error before completing, Consul-Terraform-Sync will exit with a non-zero status.
Behavior: CTS will run all tasks once with buffer periods disabled and exit. On encountering an error before completing, CTS will exit with a non-zero status.
Usage: Intended to be run before daemon-mode in order to confirm configuration is accurate and tasks update network infrastructure as expected.
## Commands
In addition to running the daemon, Consul-Terraform-Sync has a set of commands that act as a client to the daemon server. The commands provide a user-friendly experience interacting with the daemon. The commands use the Consul-Terraform-Sync's APIs but does not correspond one-to-one with it. Please see the individual commands in the left navigation for more details.
In addition to running the daemon, CTS has a set of commands that act as a client to the daemon server. The commands provide a user-friendly experience interacting with the daemon. The commands use the CTS APIs but does not correspond one-to-one with it. Please see the individual commands in the left navigation for more details.
To get help for a command, run: `consul-terraform-sync <command> -h`
### CLI Structure
Consul-Terraform-Sync commands follow the below structure
CTS commands follow the below structure
```shell-session
consul-terraform-sync <command> [options] [args]
@ -75,19 +75,19 @@ consul-terraform-sync <command> [options] [args]
Example:
```shell-session
consul-terraform-sync task disable -port=2000 task_a
consul-terraform-sync task disable -http-addr=http://localhost:2000 task_a
```
### General Options
Below are options that can be used across all commands:
| Option | Description | Required | Default |
|----------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------|--------------------|
| `-port` | Integer value that specifies the port from which the Consul-Terraform-Sync daemon serves its API.<br/>The value is prepended with `http://localhost:`, but you can specify a different scheme or address with the `-http-addr` if necessary. | Optional | `8558` |
| `-http-addr` | String value that specifies the address and port of the Consul-Terraform-Sync API. You can specify an IP or DNS address.<br/><br/> Alternatively, you can specify a value using the `CTS_HTTP_ADDR` environment variable. | Optional | `http://localhost:8558` |
| `-ssl-verify` | Boolean value that configures Consul-Terraform-Sync to enable verification for TLS/SSL connections to the API. This does not affect insecure HTTP connections.<br/><br/>Alternatively, you can specify the value using the `CTS_SSL_VERIFY` environment variable. | Optional | `true` |
| `-ca-cert` | String value that specifies the path to a PEM-encoded certificate authority file that is used to verify TLS/SSL connections. Takes precedence over `-ca-path` if both are provided.<br/><br/>Alternatively, you can specify the value using the `CTS_CACERT` environment variable. | Optional | `""` |
| `-ca-path` | String value that specifies the path to a directory containing a PEM-encoded certificate authority file that is used to verify TLS/SSL connections.<br/><br/>Alternatively, you can specify the value using the `CTS_CAPATH` environment variable. | Optional | `""` |
| `-client-cert` &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; | String value that specifies the path to a PEM-encoded client certificate that the Consul-Terraform-Sync API requires when [`verify_incoming`](/docs/nia/configuration#verify_incoming) is set to `true` on the API.<br/><br/>Alternatively, you can specify the value using the `CTS_CLIENT_CERT` environment variable. | Optional | `""` |
| `-client-key` | String value that specifies the path to a PEM-encoded client key for the certificate configured with the `-client-cert` option. This is required if `-client-cert` is set and if [`verify_incoming`](/docs/nia/configuration#verify_incoming) is set to `true` on the Consul-Terraform-Sync API.<br/><br/>Alternatively, you can specify the value using the `CTS_CLIENT_KEY` environment variable. | Optional | `""` |
| Option | Required | Type | Description | Default |
| ------ | -------- | ---- | ------------ | --------|
| `-port`| Optional | integer | **Deprecated in Consul-Terraform-Sync 0.5.0 and will be removed in a later version.** Use `-http-addr` option instead to specify the address and port of the Consul-Terraform-Sync API.<br/><br/>Port from which the CTS daemon serves its API.<br/>The value is prepended with `http://localhost:`, but you can specify a different scheme or address with the `-http-addr` if necessary. | `8558` |
| `-http-addr` | Optional | string | Address and port of the CTS API. You can specify an IP or DNS address.<br/><br/> Alternatively, you can specify a value using the `CTS_HTTP_ADDR` environment variable. | `http://localhost:8558` |
| `-ssl-verify` | Optional | boolean | Enables verification for TLS/SSL connections to the API if set to true. This does not affect insecure HTTP connections.<br/><br/>Alternatively, you can specify the value using the `CTS_SSL_VERIFY` environment variable. | `true` |
| `-ca-cert` | Optional | string | Path to a PEM-encoded certificate authority file that is used to verify TLS/SSL connections. Takes precedence over `-ca-path` if both are provided.<br/><br/>Alternatively, you can specify the value using the `CTS_CACERT` environment variable. | none |
| `-ca-path` | Optional | string | Path to a directory containing a PEM-encoded certificate authority file that is used to verify TLS/SSL connections.<br/><br/>Alternatively, you can specify the value using the `CTS_CAPATH` environment variable. | none |
| `-client-cert` &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; | Optional | string | Path to a PEM-encoded client certificate that the CTS API requires when [`verify_incoming`](/docs/nia/configuration#verify_incoming) is set to `true` on the API.<br/><br/>Alternatively, you can specify the value using the `CTS_CLIENT_CERT` environment variable. | none |
| `-client-key` | Optional | string | Path to a PEM-encoded client key for the certificate configured with the `-client-cert` option. This is required if `-client-cert` is set and if [`verify_incoming`](/docs/nia/configuration#verify_incoming) is set to `true` on the CTS API.<br/><br/>Alternatively, you can specify the value using the `CTS_CLIENT_KEY` environment variable. | none |

149
website/content/docs/nia/cli/task.mdx
View File

@ -4,20 +4,112 @@ page_title: Task Command
description: >-
Consul-Terraform-Sync supports task commands for users to modify tasks while the daemon is running
---
# task
## task create
`task create` command creates a new task so that it will run and update task resources. The command generates and outputs a Terraform plan, similar to [inspect-mode](/docs/nia/cli/cli-overview#inspect-mode), of how resources will be modified if the task is created. The command will then ask for user approval before creating the task.
It is not to be used for updating a task and will not create a task if the task name already exists.
### Usage
`consul-terraform-sync task create [options] -task-file=<task config>`
**Options:**
In addition to [general options](/docs/nia/cli/cli-overview#general-options) this command also supports the following:
| Name | Required | Type | Description |
| ------------ | -------- | ------ | ------------------------------------------------------------------------------------------------------------------- |
| `-task-file` | Required | string | The path to the task configuration file. See [configuration information](/docs/nia/configuration#task) for details. |
| `-auto-approve` &nbsp; | Optional | boolean | Whether to skip the interactive approval of the plan before creating. |
### Example
task_example.hcl:
```hcl
task {
name = "task_a"
description = ""
enabled = true,
providers = []
module = "org/example/module"
version = "1.0.0"
variable_files = []
condition "services" {
names = ["web", "api"]
}
}
```
Shell Session:
```shell-session
$ consul-terraform-sync task create -task-file=task_example.hcl
==> Inspecting changes to resource if creating task 'task_a'...
Generating plan that Consul-Terraform-Sync will use Terraform to execute
Request ID: 1da3e8e0-87c3-069b-51a6-46903e794a76
Request Payload:
// <request payload truncated>
Plan:
Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
// <diff truncated>
Plan: 1 to add, 0 to change, 0 to destroy.
==> Creating the task will perform the actions described above.
Do you want to perform these actions for 'task_a'?
- This action cannot be undone.
- Consul-Terraform-Sync cannot guarantee Terraform will perform
these exact actions if monitored services have changed.
Only 'yes' will be accepted to approve, enter 'no' or leave blank to reject.
Enter a value: yes
==> Creating and running task 'api-task'...
The task creation request has been sent to the CTS server.
Please be patient as it may take some time to see a confirmation that this task has completed.
Warning: Terminating this process will not stop task creation.
==> Task 'task_a' created
Request ID: '78eddd74-0f08-83d6-72b2-6aaac1424fba'
```
## task enable
`task enable` command enables an existing task so that it will run and update task resources. If the task is already enabled, it will return a success. The command generates and outputs a Terraform plan, similar to [inspect-mode](/docs/nia/cli#inspect-mode), of how resources will be modified if task becomes enabled. If resources changes are detected, the command will ask for user approval before enabling the task. If no resources are detected, the command will go ahead and enable the task.
### Usage
`consul-terraform-sync task enable [options] <task_name>`
**Arguments:**
- `task_name` - (string: required) The name of the task to enable
**Options:** Currently only supporting [general options](/docs/nia/cli#general-options)
| Name | Required | Type | Description |
| ----------- | -------- | ------ | ------------------------------- |
| `task_name` | Required | string | The name of the task to enable. |
**Options:**
In addition to [general options](/docs/nia/cli/#general-options) this command also supports the following:
| Name | Required | Type | Description |
| --------------- | -------- | ------- | ------------------------------- |
| `-auto-approve` | Optional | boolean | Whether to skip the interactive approval of the plan before enabling. |
### Example
@ -25,7 +117,7 @@ description: >-
$ consul-terraform-sync task enable task_a
==> Inspecting changes to resource if enabling 'task_a'...
Generating plan that Consul Terraform Sync will use Terraform to execute
Generating plan that Consul-Terraform-Sync will use Terraform to execute
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
@ -40,7 +132,7 @@ Plan: 3 to add, 0 to change, 0 to destroy.
==> Enabling the task will perform the actions described above.
Do you want to perform these actions for 'task_a'?
- This action cannot be undone.
- Consul Terraform Sync cannot guarantee Terraform will perform
- Consul-Terraform-Sync cannot guarantee Terraform will perform
these exact actions if monitored services have changed.
Only 'yes' will be accepted to approve.
@ -57,14 +149,17 @@ Enter a value: yes
`task disable` command disables an existing task so that it will no longer run and update task resources. If the task is already disabled, it will return a success.
### Usage
`consul-terraform-sync task disable [options] <task_name>`
**Arguments:**
- `task_name` - (string: required) The name of the task to disable
| Name | Required | Type | Description |
| ----------- | -------- | ------ | -------------------------------- |
| `task_name` | Required | string | The name of the task to disable. |
**Options:** Currently only supporting [general options](/docs/nia/cli#general-options)
### Example
```shell-session
@ -73,3 +168,45 @@ $ consul-terraform-sync task disable task_b
==> 'task_b' disable complete!
```
## task delete
`task delete` command deletes an existing task. The command will ask the user for approval before deleting the task. The task will be marked for deletion and will be deleted immediately if it is not running. Otherwise, the task will be deleted once it has completed.
-> **Note:** Deleting a task will not destroy the infrastructure managed by the task.
### Usage
`consul-terraform-sync task delete [options] <task_name>`
**Arguments:**
| Name | Required | Type | Description |
| ----------- | -------- | ------ | ------------------------------- |
| `task_name` | Required | string | The name of the task to delete. |
**Options:**
In addition to [general options](/docs/nia/cli/cli-overview#general-options) this command also supports the following:
| Name | Required | Type | Description |
| --------------- | -------- | ------- | ------------------------------- |
| `-auto-approve` | Optional | boolean | Whether to skip the interactive approval of the task deletion. |
### Example
```shell-session
$ consul-terraform-sync task delete task_a
==> Do you want to delete 'task_a'?
- This action cannot be undone.
- Deleting a task will not destroy the infrastructure managed by the task.
- If the task is not running, it will be deleted immediately.
- If the task is running, it will be deleted once it has completed.
Only 'yes' will be accepted to approve, enter 'no' or leave blank to reject.
Enter a value: yes
==> Marking task 'task_a' for deletion...
==> Task 'task_a' has been marked for deletion and will be deleted when not running.
```

354
website/content/docs/nia/configuration.mdx
View File

@ -7,11 +7,11 @@ description: >-
# Configuration Options for Consul-Terraform-Sync
The Consul-Terraform-Sync daemon is configured using configuration files and supports [HashiCorp Configuration Language](https://github.com/hashicorp/hcl) (HCL) and JSON file formats.
The Consul-Terraform-Sync (CTS) daemon is configured using configuration files and supports [HashiCorp Configuration Language](https://github.com/hashicorp/hcl) (HCL) and JSON file formats.
## Global Config Options
Top level options are reserved for configuring Consul-Terraform-Sync.
Top level options are reserved for configuring CTS.
```hcl
log_level = "INFO"
@ -41,27 +41,27 @@ tls {
- `enabled` - (bool: true) Enable or disable buffer periods globally. Specifying `min` will also enable it.
- `min` - (string: "5s") The minimum period of time to wait after changes are detected before triggering related tasks.
- `max` - (string: "20s") The maximum period of time to wait after changes are detected before triggering related tasks. If `min` is set, the default period for `max` is 4 times the value of `min`.
- `log_level` - (string: "INFO") The log level to use for Consul-Terraform-Sync logging. This defaults to "INFO". The available log levels are "TRACE", "DEBUG", "INFO", "WARN", and "ERR".
- `port` - (int: 8558) The port for Consul-Terraform-Sync to use to serve API requests.
- `log_level` - (string: "INFO") The log level to use for CTS logging. This defaults to "INFO". The available log levels are "TRACE", "DEBUG", "INFO", "WARN", and "ERR".
- `port` - (int: 8558) The port for CTS to use to serve API requests.
- `syslog` - Specifies the syslog server for logging.
- `enabled` - (bool) Enable syslog logging. Specifying other option also enables syslog logging.
- `facility` - (string: "local0") Name of the syslog facility to log to.
- `name` - (string: "consul-terraform-sync") Name to use for the daemon process when logging to syslog.
- `working_dir` - (string: "sync-tasks") Specifies the base working directory for managing the artifacts generated by Consul-Terraform-Sync for each task, including Terraform configuration files. Consul-Terraform-Sync will also generate a subdirectory for each task, e.g., `./sync-tasks/task-name`. The subdirectory is the working directory for the task. Use the [`task.working_dir`](#working_dir-1) option to override task-level working directories.
- `tls` - Configure TLS on the Consul-Terraform-Sync API.
- `working_dir` - (string: "sync-tasks") Specifies the base working directory for managing the artifacts generated by CTS for each task, including Terraform configuration files. CTS will also generate a subdirectory for each task, e.g., `./sync-tasks/task-name`. The subdirectory is the working directory for the task. Use the [`task.working_dir`](#working_dir-1) option to override task-level working directories.
- `tls` - Configure TLS on the CTS API.
- `enabled` - (bool: false) Enable TLS. Providing a value for any of the TLS options will enable this parameter implicitly.
- `cert` - (string) The path to a PEM-encoded certificate file used for TLS connections to the CTS API.
- `key` - (string) The path to the PEM-encoded private key file used with the certificate configured by `cert`.
- `verify_incoming` - (bool: false) Enable mutual TLS. Requires all incoming connections to the CTS API to use a TLS connection and provide a certificate signed by a Certificate Authority specified by the `ca_cert` or `ca_path`.
- `ca_cert` - (string) The path to a PEM-encoded certificate authority file used to verify the authenticity of the incoming client connections to the CTS API when `verify_incoming` is set to true. Takes precedence over `ca_path` if both are configured.
- `ca_path` - (string) The path to a directory of PEM-encoded certificate authority files used to verify the authenticity of the incoming client connections to the CTS API when `verify_incoming` is set to true.
- `license_path` <EnterpriseAlert inline /> - (string: "") Configures the path to the file containing the license. A license must be set to use enterprise features. You can also set the license by defining the `CONSUL_LICENSE` and `CONSUL_LICENSE_PATH` environment variables. See [Setting the License](/docs/nia/enterprise/license#setting-the-license) for details.
- `license_path` <EnterpriseAlert inline /> - (string) Configures the path to the file containing the license. A license must be set to use enterprise features. You can also set the license by defining the `CONSUL_LICENSE` and `CONSUL_LICENSE_PATH` environment variables. See [Setting the License](/docs/nia/enterprise/license#setting-the-license) for details.
## Consul
The `consul` block is used to configure Consul-Terraform-Sync connection with a Consul agent to perform queries to the Consul Catalog and Consul KV pertaining to task execution.
The `consul` block is used to configure CTS connection with a Consul agent to perform queries to the Consul Catalog and Consul KV pertaining to task execution.
-> **Note:** Use HTTP/2 to improve Consul-Terraform-Sync performance when communicating with the local Consul process. [TLS/HTTPS](/docs/agent/options) must be configured for the local Consul with the [cert_file](/docs/agent/options#cert_file) and [key_file](/docs/agent/options#key_file) parameters set. For the Consul-Terraform-Sync configuration, set `tls.enabled = true` and set the `address` parameter to the HTTPS URL, e.g., `address = example.consul.com:8501`. If using self-signed certificates for Consul, you will also need to set `tls.verify = false` or add the certificate to `ca_cert` or `ca_path`.
-> **Note:** Use HTTP/2 to improve CTS performance when communicating with the local Consul process. [TLS/HTTPS](/docs/agent/options) must be configured for the local Consul with the [cert_file](/docs/agent/options#cert_file) and [key_file](/docs/agent/options#key_file) parameters set. For the CTS configuration, set `tls.enabled = true` and set the `address` parameter to the HTTPS URL, e.g., `address = example.consul.com:8501`. If using self-signed certificates for Consul, you will also need to set `tls.verify = false` or add the certificate to `ca_cert` or `ca_path`.
To read more on suggestions for configuring the Consul agent, see [run an agent](/docs/nia/installation/requirements#run-an-agent).
@ -80,16 +80,16 @@ consul {
- `enabled` - (bool)
- `username` - (string)
- `password` - (string)
- `tls` - Configure TLS to use a secure client connection with Consul. Using HTTP/2 can solve issues related to hitting Consul's maximum connection limits, as well as improve efficiency when processing many blocking queries. This option is required for Consul-Terraform-Sync when connecting to a [Consul agent with TLS verification enabled for HTTPS connections](/docs/agent/options#verify_incoming).
- `tls` - Configure TLS to use a secure client connection with Consul. Using HTTP/2 can solve issues related to hitting Consul's maximum connection limits, as well as improve efficiency when processing many blocking queries. This option is required for CTS when connecting to a [Consul agent with TLS verification enabled for HTTPS connections](/docs/agent/options#verify_incoming).
- `enabled` - (bool) Enable TLS. Providing a value for any of the TLS options will enable this parameter implicitly.
- `verify` - (bool: true) Enables TLS peer verification. The default is enabled, which will check the global certificate authority (CA) chain to make sure the certificates returned by Consul are valid.
- `verify` - (bool: true) Enables TLS peer verification. The default is enabled, which will check the global certificate authority (CA) chain to make sure the certificates returned by Consul are valid.
- If Consul is using a self-signed certificate that you have not added to the global CA chain, you can set this certificate with `ca_cert` or `ca_path`. Alternatively, you can disable SSL verification by setting `verify` to false. However, disabling verification is a potential security vulnerability.
- `ca_cert` - (string) The path to a PEM-encoded certificate authority file used to verify the authenticity of the connection to Consul over TLS. Can also be provided through the `CONSUL_CACERT` environment variable.
- `ca_path` - (string) The path to a directory of PEM-encoded certificate authority files used to verify the authenticity of the connection to Consul over TLS. Can also be provided through the `CONSUL_CAPATH` environment variable.
- `cert` - (string) The path to a PEM-encoded client certificate file provided to Consul over TLS in order for Consul to verify the authenticity of the connection from CTS. Required if Consul has `verify_incoming` set to true. Can also be provided through the `CONSUL_CLIENT_CERT` environment variable.
- `key` - (string) The path to the PEM-encoded private key file used with the client certificate configured by `cert`. Required if Consul has `verify_incoming` set to true. Can also be provided through the `CONSUL_CLIENT_KEY` environment variable.
- `server_name` - (string) The server name to use as the Server Name Indication (SNI) for Consul when connecting via TLS. Can also be provided through the `CONSUL_TLS_SERVER_NAME` environment variable.
- `token` - (string) The ACL token to use for client communication with the local Consul agent. The token can also be provided through the `CONSUL_TOKEN` or `CONSUL_HTTP_TOKEN` environment variables. More information on the required privileges required by Consul-Terraform-Sync are available in the [Secure Consul-Terraform-Sync for Production](https://learn.hashicorp.com/tutorials/consul/consul-terraform-sync-secure?utm_source=WEBSITE&utm_medium=WEB_IO&utm_offer=ARTICLE_PAGE&utm_content=DOCS#configure-acl-privileges-for-consul-terraform-sync) tutorial
- `token` - (string) The ACL token to use for client communication with the local Consul agent. The token can also be provided through the `CONSUL_TOKEN` or `CONSUL_HTTP_TOKEN` environment variables. More information on the required privileges required by CTS are available in the [Secure Consul-Terraform-Sync for Production](https://learn.hashicorp.com/tutorials/consul/consul-terraform-sync-secure?utm_source=WEBSITE&utm_medium=WEB_IO&utm_offer=ARTICLE_PAGE&utm_content=DOCS#configure-acl-privileges-for-consul-terraform-sync) tutorial
- `transport` - Transport configures the low-level network connection details.
- `dial_keep_alive` - (string: "30s") The amount of time for keep-alives.
- `dial_timeout` - (string: "30s") The amount of time to wait to establish a connection.
@ -98,12 +98,16 @@ consul {
- `max_idle_conns` - (int: 0) The maximum number of total idle connections across all hosts. The limit is disabled by default.
- `max_idle_conns_per_host` - (int: 100) The maximum number of idle connections per remote host. The majority of connections are established with one host, the Consul agent.
- To achieve the shortest latency between a Consul service update to a task execution, configure `max_idle_conns_per_host` equal to or greater than the number of services in automation across all tasks.
- This value should be lower than the configured [`http_max_conns_per_client`](/docs/agent/options#http_max_conns_per_client) for the Consul agent. If `max_idle_conns_per_host` and the number of services in automation is greater than the Consul agent limit, Consul-Terraform-Sync may error due to connection limits (status code 429). You may increase the agent limit with caution. _Note: requests to the Consul agent made by Terraform subprocesses or any other process on the same host as Consul-Terraform-Sync will contribute to the Consul agent connection limit._
- This value should be lower than the configured [`http_max_conns_per_client`](/docs/agent/options#http_max_conns_per_client) for the Consul agent. If `max_idle_conns_per_host` and the number of services in automation is greater than the Consul agent limit, CTS may error due to connection limits (status code 429). You may increase the agent limit with caution. _Note: requests to the Consul agent made by Terraform subprocesses or any other process on the same host as CTS will contribute to the Consul agent connection limit._
- `tls_handshake_timeout` - (string: "10s") amount of time to wait to complete the TLS handshake.
## Service
A `service` block is an optional block to explicitly define configuration of services that Consul-Terraform-Sync monitors. A `service` block is only necessary for services that have non-default values e.g. custom datacenter. Services that do not have a `service` block configured will assume default values. To configure multiple services, specify multiple `service` blocks. For services to be included in task automation, the service must be included in the `task.services` field of a [`task` block](#task). If a `service` block is configured, the service can be referred in `task.services` by service name or ID. If a `service` block is not configured, it can only be referred to by service name.
~> Deprecated in CTS 0.5.0 and will be removed in a future major release. `service` blocks are used to define the `task` block's `services` fields, which were also deprecated and replaced with [Services Condition](/docs/nia/configuration#services-condition) and [Services Module Input](/docs/nia/configuration#services-module-input). `service` block configuration can be replaced by configuring the equivalent fields of the corresponding Services Condition and Services Module Input. See [0.5.0 release notes](/docs/nia/release-notes/0-5-0#deprecate-service-block) for examples.
A `service` block is an optional block to explicitly define the services configured in the `task` block's `services` field (deprecated). `service` blocks do not define services configured in the `task` block's `condition "services"` or `module_input "services` blocks.
A `service` block is only necessary for services that have non-default values e.g. custom datacenter. Services that do not have a `service` block configured will assume default values. To configure multiple services, specify multiple `service` blocks. If a `service` block is configured, the service can be referred in `task.services` by service name or ID. If a `service` block is not configured, it can only be referred to by service name.
```hcl
service {
@ -113,18 +117,19 @@ service {
}
```
- `datacenter` - (string) The datacenter the service is deployed in.
- `description` - (string) The human readable text to describe the service.
- `id` - (string) ID identifies the service for Consul-Terraform-Sync. This is used to explicitly identify the service config for a task to use. If no ID is provided, the service is identified by the service name within a [task definition](#task).
- `name` - (string: required) The Consul logical name of the service (required).
- `namespace` <EnterpriseAlert inline /> - (string: "default") The namespace of the service. If not provided, the namespace will be inferred from the Consul-Terraform-Sync ACL token, or default to the `default` namespace.
- `filter` - (string) Specifies the expression used to filter nodes for the service. For more details on supported filters, see the Consul documentation on [filtering service nodes](/api-docs/health#filtering-2).
- `cts_user_defined_meta` - (map[string]) User-defined metadata is a map of strings that will be appended to the [service input variable](/docs/nia/installation/requirements#module-specifications) for compatible Terraform modules. Not all modules may use this value. To determine if your task uses metadata or what the expected keys and format are, reference documentation for the module(s) configured for your tasks.
- If multiple tasks depend on the same service but require different metadata, you can declare different sets of metadata for the same service. Define multiple service blocks for the service with unique IDs (and identical names) for those blocks. The metadata can then be separated per task based on the service IDs.
| Parameter | Required | Type | Description | Default |
| --------- | -------- | ---- | ----------- | ------- |
| `name` | Required | string | Consul logical name of the service. | none |
| `id` | Optional | string | Service ID for CTS. This is used to explicitly identify the service config for a task to use. If no ID is provided, the service is identified by the service name within a [task definition](#task). | none |
| `description` | Optional | string | Human-readable text to describe the service | none |
| `datacenter` | Optional | string | Name of a datacenter to query for the task. | Datacenter of the agent that CTS queries. |
| `namespace` | Optional | string |<EnterpriseAlert inline /> <br/> Namespace of the services to query for the task. | In order of precedence: <br/> 1. Inferred from the CTS ACL token <br/> 2. The `default` namespace. |
| `filter` | Optional | string | Expression used to additionally filter the services to monitor. <br/><br/> Refer to the [services filtering documentation](/api-docs/health#filtering-2) and section about [how to write filter expressions](/api-docs/features/filtering#creating-expressions) for additional information. | none |
| `cts_user_defined_meta` | Optional | map[string] | User-defined metadata that is appended to the [service input variable](/docs/nia/terraform-modules#services-module-input) for compatible Terraform modules. <br/><br/> Some modules do not use the configured metadata. Refer to the module configured for the task for information about metadata usage and expected keys and format. <br/><br/> If multiple tasks depend on the same service but require different metadata, you can declare different sets of metadata for the same service. Define multiple service blocks for the service with unique IDs (and identical names) for those blocks. The metadata can then be separated per task based on the service IDs. | none|
## Task
A `task` block configures which task to execute in automation. When the task should execute can be determined by the `service` block and `condition` block. The `task` block may be specified multiple times to configure multiple tasks.
A `task` block configures which task to execute in automation. When the task should execute can be determined by the `service` block (deprecated) and `condition` block. The `task` block may be specified multiple times to configure multiple tasks.
```hcl
task {
@ -132,35 +137,40 @@ task {
description = ""
enabled = true,
providers = []
services = ["web", "api"]
source = "org/example/module"
module = "org/example/module"
version = "1.0.0"
variable_files = []
condition "catalog-services" {
regexp = ".*"
condition "services" {
names = ["web", "api"]
}
}
```
- `description` - (string) The human readable text to describe the service.
- `description` - (string) The human readable text to describe the task.
- `name` - (string: required) Name is the unique name of the task (required). A task name must start with a letter or underscore and may contain only letters, digits, underscores, and dashes.
- `enabled` - (bool: true) Enable or disable a task from running and managing resources.
- `providers` - (list[string]) Providers is the list of provider names the task is dependent on. This is used to map [Terraform provider configuration](#terraform-provider) to the task.
- `services` - (list[string]) Required depending on [`condition`](#condition) configuration. Services is the list of logical service names or service IDs the task executes on. Consul-Terraform-Sync monitors the Consul Catalog for changes to these services and triggers the task to run. Any service value not explicitly defined by a `service` block with a matching ID is assumed to be a logical service name in the default namespace. Alternative to configuring `services`, a `condition` can be configured so that the task does not trigger on changes to services (default behavior) but instead trigger on a different condition. See [Task Condition](#task-condition) configuration for more details.
- `source` - (string: required) Source is the location the driver uses to discover the Terraform module used for automation. The source is the module path which can be local or remote on the [Terraform Registry](https://registry.terraform.io/) or private module registry. Read more on [Terraform module source and other supported types here](https://www.terraform.io/docs/modules/sources.html).
- `services` - (list[string]) **Deprecated in CTS 0.5.0 and will be removed in a future major release. Use [Services Condition](/docs/nia/configuration#services-condition) or [Services Module Input](/docs/nia/configuration#services-module-input) instead. See [0.5.0 release notes](/docs/nia/release-notes/0-5-0#deprecate-services-field) for examples.** Specifies an optional list of logical service names or service IDs that the task monitors for changes in the Consul catalog. The `services` can act in different ways depending on the configuration of the task's `condition` block:
- no `condition` block configured: `services` will act as the task's condition and provide the services information as module input
- the `condition` block configured for type `services`: `services` is incompatible with this type of `condition` because both configure the services module input. CTS will return an error.
- the `condition` block configured for all other types: `services` will act only to provide services module input.
- To use a private module with the [`terraform` driver](#terraform-driver), run the command [`terraform login [hostname]`](https://learn.hashicorp.com/tutorials/terraform/cloud-login) to authenticate the local Terraform CLI prior to starting Consul-Terraform-Sync.
Service values that are not explicitly defined by a `service` block that have a matching ID are assumed to be logical service names in the `default` namespace.
- `source` - (string: required) **Deprecated in CTS 0.5.0 and will be removed in a future major release. See the `module` field instead.**
- `module` - (string: required) Module is the location the driver uses to discover the Terraform module used for automation. The module's source can be local or remote on the [Terraform Registry](https://registry.terraform.io/) or private module registry. Read more on [Terraform module source and other supported types here](https://www.terraform.io/docs/modules/sources.html).
- To use a private module with the [`terraform` driver](#terraform-driver), run the command [`terraform login [hostname]`](https://learn.hashicorp.com/tutorials/terraform/cloud-login) to authenticate the local Terraform CLI prior to starting CTS.
- To use a private module with the [`terraform_cloud` driver](#terraform-cloud-driver), no extra steps are needed.
```hcl
// local module example: "./terraform-cts-hello"
source = "<PATH>"
module = "<PATH>"
// public module example: "mkam/hello/cts"
source = "<NAMESPACE>/<MODULE NAME>/<PROVIDER>"
module = "<NAMESPACE>/<MODULE NAME>/<PROVIDER>"
// private module example: "my.tfe.hostname.io/my-org/hello/cts"
source = "<HOSTNAME>/<ORGANIZATION>/<MODULE NAME>/<PROVIDER>"
module = "<HOSTNAME>/<ORGANIZATION>/<MODULE NAME>/<PROVIDER>"
```
- `variable_files` - (list[string]) Specifies list of paths to [Terraform variable definition files (`.tfvars`)](https://www.terraform.io/docs/configuration/variables.html#variable-definitions-tfvars-files). The content of these files should consist of only variable name assignments. The variable assignments must match the corresponding variable declarations made available by the Terraform module for the task.
@ -178,40 +188,81 @@ task {
</CodeBlockConfig>
- `version` - (string) The version of the provided source the task will use. For the [Terraform driver](#terraform-driver), this is the module version. The latest version will be used as the default if omitted.
- `working_dir` - (string) The working directory to manage generated artifacts by Consul-Terraform-Sync for this task, including Terraform configuration files. By default, a working directory is created for each task as a subdirectory in the base [`working_dir`](#working_dir), e.g. `sync-tasks/task-name`.
- `version` - (string) The version of the provided module the task will use. The latest version will be used as the default if omitted.
- `working_dir` - (string) The working directory to manage generated artifacts by CTS for this task, including Terraform configuration files. By default, a working directory is created for each task as a subdirectory in the base [`working_dir`](#working_dir), e.g. `sync-tasks/task-name`.
- `buffer_period` - Configures the buffer period for a dynamic task to dampen the effects of flapping services to downstream network devices. It defines the minimum and maximum amount of time to wait for the cluster to reach a consistent state and accumulate changes before triggering task execution. The default is inherited from the top level [`buffer_period` block](#global-config-options). If configured, these values will take precedence over the global buffer period. This is useful to enable for a task that is dependent on services that have a lot of flapping. Buffer periods do not apply to scheduled tasks.
- `enabled` - (bool) Enable or disable buffer periods for this task. Specifying `min` will also enable it.
- `min` - (string: "5s") The minimum period of time to wait after changes are detected before triggering related tasks.
- `max` - (string: "20s") The maximum period of time to wait after changes are detected before triggering related tasks. If `min` is set, the default period for `max` is 4 times the value of `min`.
- `condition` - (obj) The requirement that, when met, triggers Consul-Terraform-Sync to execute the task. When unconfigured, the default condition is to trigger the task on changes in the services configured in [`services`](#services). Only one `condition` may be configured per task. Consul-Terraform-Sync supports different types of conditions, which each have their own configuration options. See [Task Condition](#task-condition) configuration for full details on configuration options for each condition type.
- `source_input` - (obj) Specifies a Consul object containing values or metadata to be provided to the Terraform Module. When source input is empty, source input will be determined by the condition or services list. Only one `source_input` may be configured per task. Consul-Terraform-Sync supports different types of source input, each source input has their own configuration options. The source input block is currently only supported with [schedule condition](#schedule-condition). See [Task Source Input](#task-source-input) configuration for full details on configuration options for each source input type.
- `condition` - (obj: required) The requirement that, when met, triggers CTS to execute the task. Only one `condition` may be configured per task. CTS supports different types of conditions, which each have their own configuration options. See [Task Condition](#task-condition) configuration for full details on configuration options for each condition type.
- `source_input` - (obj) **Deprecated in CTS 0.5.0 and will be removed in 0.8.0. See the `module_input` block instead.**
- `module_input` - (obj) Specifies a Consul object containing values or metadata to be provided to the Terraform Module. The `module_input` block defines any extra module inputs needed for task execution. This is in addition to any module input provided by the `condition` block or `services` field (deprecated). Multiple `module_input` blocks can be configured per task. [Task Module Input](#task-module-input) configuration for full details on usage and restrictions.
- `terraform_version` - (string) <EnterpriseAlert inline /> The version of Terraform to use for the Terraform Cloud workspace associated with the task. Defaults to the latest compatible version supported by the organization. This option is only available when used with the [Terraform Cloud driver](#terraform-cloud-driver); otherwise, set the version within the [Terraform driver](#terraform-driver).
### Task Condition
A `task` block can be optionally configured with a `condition` block to set the conditions that should be met in order to execute that particular task. Below are the different types of conditions that Consul-Terraform-Sync supports.
A `task` block is configured with a `condition` block to set the conditions that should be met in order to execute that particular task. Below are the different types of conditions that CTS supports.
#### Services Condition
This is the default type of condition when no `condition` block is configured for a task. This condition will trigger the task on either changes in services configured in [`task.services`](#services) or changes in services that match the regular expression configured in `regexp`. The default behavior is to trigger on changes to [`task.services`](#services). The [`task.services`](#services) option is required if `regexp` is not configured.
This condition will trigger the task on services that match the regular expression configured in `regexp` or services listed by name in `names`. Either `regexp` or `names` must be configured, but not both.
When a `condition "services"` block is configured for a task, then the following restrictions become applicable:
- the task cannot be configured with the `services` field (deprecated)
- the task cannot be configure with a `module_input "services"` or `source_input "services"` (deprecated) block
These restrictions are due to the fact that the monitored services information for a task can only be set through one configuration option. Any services module input that the task needs should be configured solely through the `condition` block.
See [Task Execution: Services Condition](/docs/nia/tasks#services-condition) for more details on how tasks are triggered with a services condition.
```hcl
task {
name = "services_condition_task"
name = "services_condition_regexp_task"
description = "execute on changes to services with names starting with web"
providers = ["my-provider"]
source = "path/to/services-condition-module"
module = "path/to/services-condition-module"
condition "services" {
regexp = "^web.*"
regexp = "^web.*"
datacenter = "dc1"
namespace = "default"
filter = "Service.Tags not contains \"prod\""
cts_user_defined_meta {
key = "value"
}
}
}
```
```hcl
task {
name = "services_condition_names_task"
description = "execute on changes to services with names api or web"
module = "path/to/services-condition-module"
condition "services" {
names = ["api", "web"]
datacenter = "dc1"
namespace = "default"
filter = "Service.Tags not contains \"prod\""
cts_user_defined_meta {
key = "value"
}
}
}
```
| Parameter | Required | Type | Description | Default |
| --------- | -------- | ---- | ----------- | ------- |
| `regexp` | Required if `names` is not configured | string | Regular expression used to match the names of Consul services to monitor. Only services that have a name matching the regular expression are used by the task. <br/><br/> If both a list and a regex are needed, consider including the list as part of the regex or creating separate tasks. | none |
| `names` | Required if `regexp` is not configured | list[string] | Names of Consul services to monitor. Only services that have their name listed in `names` are used by the task. | none |
| `datacenter` | Optional | string | Name of a datacenter to query for the task. | Datacenter of the agent that CTS queries. |
| `namespace` | Optional | string | <EnterpriseAlert inline /> <br/> Namespace of the services to query for the task. | In order of precedence: <br/> 1. Inferred from the CTS ACL token <br/> 2. The `default` namespace. |
| `filter` | Optional | string | Expression used to additionally filter the services to monitor. <br/><br/> Refer to the [services filtering documentation](/api-docs/health#filtering-2) and section about [how to write filter expressions](/api-docs/features/filtering#creating-expressions) for additional information. | none |
| `cts_user_defined_meta` | Optional | map[string] | User-defined metadata that is appended to the [service input variable](/docs/nia/terraform-modules#services-module-input) for compatible Terraform modules. <br/><br/> Some modules do not use the configured metadata. Refer to the module configured for the task for information about metadata usage and expected keys and format. | none|
| `source_includes_var` | Optional | boolean | **Deprecated in CTS 0.5.0 and will be removed in 0.8.0. See the `use_as_module_input` field instead.** | true |
| `use_as_module_input` | Optional | boolean | Whether or not the values of the condition object should also be used as input for the [`services` variable](/docs/nia/terraform-modules#services-variable) for the Terraform module<br/><br/> Please refer to the selected module's documentation for guidance on how to configure this field. If configured inconsistently with the module, CTS will error and exit. | true |
- `regexp` - **(beta)** (string) Only services that have a name which matches the regular expression are used by the task. If `regexp` is configured, then [`task.services`](#services) must be omitted or empty. If both a list and a regex are needed, consider including the list as part of the regex or creating separate tasks.
#### Catalog-Services Condition
@ -223,18 +274,14 @@ See [Task Execution: Catalog Services Condition](/docs/nia/tasks#catalog-service
task {
name = "catalog_service_condition_task"
description = "execute on service de/registrations with name matching 'web.*'"
source = "path/to/catalog-services-module"
module = "path/to/catalog-services-module"
providers = ["my-provider"]
// configure depending on module. provides detailed information for these
// services but does not execute task. refer to module docs on how to configure.
services = ["web-api"]
condition "catalog-services" {
datacenter = "dc1"
namespace = "default"
regexp = "web.*"
source_includes_var = false
use_as_module_input = true
node_meta {
key = "value"
}
@ -242,15 +289,21 @@ task {
}
```
- `datacenter` - (string) The datacenter of the services to query for the task. If not provided, the datacenter will default to the datacenter of the agent that Consul-Terraform-Sync queries.
- `namespace` <EnterpriseAlert inline /> - (string) The namespace of the services to query for the task. If not provided, the namespace will be inferred from the Consul-Terraform-Sync ACL token or default to the `default` namespace.
- `node_meta` - (map[string]) The node metadata key/value pairs used to filter services. Only services registered at a node with the specified key/value pairs are used by the task.
- `regexp` - (string) Optional if [`task.services`](/docs/nia/configuration#services) is configured. Either `regexp` or `task.services` or both must be configured. Only services that have a name which matches the regular expression are used by the task. If not provided, `regexp` will default to an exact match on `task.services` list. For example, if `task.services = ["api", "web"]`, then `regexp` will default to `^api$|^web$`. See [Task Execution: Catalog Services Condition](/docs/nia/tasks#catalog-services-condition) for more details on the relationship between `task.services` and `regexp`. Some resources for more information on regular expressions: [regular expression syntax](https://github.com/google/re2/wiki/Syntax), [try out regular expression string matching](https://golang.org/pkg/regexp/#Regexp.MatchString).
- `source_includes_var` - (bool: false) Whether or not the module configured at [`task.source`](#source) includes the [`catalog_services` variable](/docs/nia/terraform-modules#catalog-services-variable). Please refer to the documentation of the selected module for guidance on how to configure this field. If configured inconsistently with the module, Consul-Terraform-Sync will error and exit.
| Parameter | Required | Type | Description | Default |
| --------- | -------- | ---- | ----------- | ------- |
| `regexp` | Required | string | Regular expression used to match the names of Consul service to monitor for registration and deregistration. Only services that have a name matching the regular expression are used by the task. <br/><br/> Refer to [regular expression syntax documentation](https://github.com/google/re2/wiki/Syntax) and [try out regular expression string matching](https://golang.org/pkg/regexp/#Regexp.MatchString) for additional information. | none |
| `datacenter` | Optional | string | Name of a datacenter to query for the task. | Datacenter of the agent that CTS queries. |
| `namespace` | Optional | string | <EnterpriseAlert inline /> <br/> Namespace of the services to query for the task. | In order of precedence: <br/> 1. Inferred from the CTS ACL token <br/> 2. The `default` namespace. |
| `node_meta` | Optional | map[string] | Node metadata key/value pairs to use to filter services. Only services registered at a node with the specified key/value pairs are used by the task. | none |
| `source_includes_var` | Optional | boolean | **Deprecated in CTS 0.5.0 and will be removed in 0.8.0. See the `use_as_module_input` field instead.** | true |
| `use_as_module_input` | Optional | boolean | Whether or not the values of the condition object should also be used as input for the [`catalog_services` variable](/docs/nia/terraform-modules#catalog-services-variable) for the Terraform module<br/><br/> Please refer to the selected module's documentation for guidance on how to configure this field. If configured inconsistently with the module, CTS will error and exit. | true |
#### Consul KV Condition
A consul-kv condition block configures a task to only execute on changes to a Consul KV entry. The condition can be configured for a single Consul KV entry or for any Consul KV entries that are prefixed with a given path.
A `condition "consul-kv"` block configures a task to only execute on changes to a Consul KV entry. The condition can be configured for a single Consul KV entry or for any Consul KV entries that are prefixed with a given path.
When a `condition "consul-kv"` block is configured for a task, the task cannot be configured with a `module_input "consul-kv"` or `source_input "consul-kv"` (deprecated) block. The monitored consul-kv information for a task can only be set through one configuration option. Any consul-kv module input that the task needs should be configured solely through the `condition` block.
See [Task Execution: Consul KV Condition](/docs/nia/tasks#consul-kv-condition) for more information on how tasks are triggered with a consul-kv condition.
@ -258,63 +311,99 @@ See [Task Execution: Consul KV Condition](/docs/nia/tasks#consul-kv-condition) f
task {
name = "consul_kv_condition_task"
description = "execute on changes to Consul KV entry"
source = "path/to/consul-kv-module"
module = "path/to/consul-kv-module"
providers = ["my-provider"]
services = ["web-api"]
condition "consul-kv" {
path = "my-key"
recurse = false
datacenter = "dc1"
namespace = "default"
source_includes_var = false
use_as_module_input = true
}
}
```
- `path` - (string: required) The path of the key that is used by the task.
- `recurse` - (bool: false) Setting to `true` instructs Consul-Terraform-Sync to treat the path as a prefix instead of a literal match.
- `datacenter` - (string) The datacenter of the services to query for the task. If not provided, the datacenter will default to the datacenter of the agent that Consul-Terraform-Sync queries.
- `namespace` <EnterpriseAlert inline /> - (string) The namespace of the services to query for the task. If not provided, the namespace will be inferred from the Consul-Terraform-Sync ACL token or default to the `default` namespace.
- `source_includes_var` - (bool: false) If set to `true`, then Consul-Terraform-Sync will include the [`consul_kv` variable](/docs/nia/terraform-modules#consul-kv-variable) as an input to the module specified in `task.source`. Refer to the documentation of the selected module for guidance on how to configure this field. If configured inconsistently with the module, Consul-Terraform-Sync will error and exit.
| Parameter | Required | Type | Description | Default |
| --------- | -------- | ---- | ----------- | ------- |
| `path` | Required | string | Path of the key used by the task. The path can point to a single Consul KV entry or several entries within the path. | none |
| `recurse` | Optional | boolean | Enables CTS to treat the path as a prefix. If set to `false`, the path will be treated as a literal match. | `false` |
| `datacenter` | Optional | string | Name of a datacenter to query for the task. | Datacenter of the agent that CTS queries. |
| `namespace` | Optional | string | <EnterpriseAlert inline /> <br/> Namespace of the services to query for the task. | In order of precedence: <br/> 1. Inferred from the CTS ACL token <br/> 2. The `default` namespace. |
| `source_includes_var` | Optional | boolean | **Deprecated in CTS 0.5.0 and will be removed in 0.8.0. See the `use_as_module_input` field instead.** | true |
| `use_as_module_input` | Optional | boolean | Whether or not the values of the condition object should also be used as input for the [`consul_kv` variable](/docs/nia/terraform-modules#consul-kv-variable) for the Terraform module<br/><br/> Please refer to the selected module's documentation for guidance on how to configure this field. If configured inconsistently with the module, CTS will error and exit. | true |
#### Schedule Condition
A scheduled task has a schedule condition block, which defines the schedule for executing the task. Unlike a dynamic task, a scheduled task does not dynamically trigger on changes in Consul.
Schedule tasks also rely on additional task configuration, separate from the condition block to determine the source input information to provide to the task module. See [`task.services`](#services) or [`source_input`](#source_input) block configuration for details on how to configure source input.
Schedule tasks also rely on additional task configuration, separate from the condition block to determine the module input information to provide to the task module. See [`module_input`](#module_input) block configuration for details on how to configure module input.
See [Task Execution: Schedule Condition](/docs/nia/tasks#schedule-condition) for more information on how tasks are triggered with schedule conditions.
See [Terraform Module: Source Input](/docs/nia/terraform-modules#source-input) for more information on source input options for a scheduled task.
See [Terraform Module: Module Input](/docs/nia/terraform-modules#module-input) for more information on module input options for a scheduled task.
```hcl
task {
name = "scheduled_task"
description = "execute every Monday using service information from web and db"
services = ["web", "db"]
source = "path/to/module"
module = "path/to/module"
condition "schedule" {
cron = "* * * * Mon"
}
module_input "services" {
names = ["web", "db"]
}
}
```
- `cron` - (string: required) The CRON expression that dictates the schedule to trigger the task. For more information on CRON expressions, see the [cronexpr parsing library](https://github.com/hashicorp/cronexpr).
| Parameter | Required | Type | Description | Default |
| --------- | -------- | ---- | ----------- | ------- |
| `cron` | Required | string | The CRON expression that dictates the schedule to trigger the task. For more information on CRON expressions, see the [cronexpr parsing library](https://github.com/hashicorp/cronexpr). | none |
### Task Source Input
### Task Module Input ((#task-source-input))
You can add an optional `source_input` block to the `task` block. The `source_input` block specifies a Consul object containing values or metadata to be provided to the Terraform Module. Consul-Terraform-Sync supports the following source input types.
~> `module_input` was renamed from `source_input` in CTS 0.5.0. Documentation for the `module_input` block also applies to the `source_input` block.
~> **The source input block is currently only supported when using a schedule condition.** Adding a `source_input` block alongside any other type of condition will result in an error. To accomplish a similar behavior with other condition blocks, use the `source_includes_var` field.
You can optionally add one or more `module_input` blocks to the `task` block. A `module_input` block specifies a Consul object containing values or metadata to be provided to the Terraform Module. Both scheduled and dynamic tasks can be configured with `module_input` blocks.
#### Services Source Input
The example below shows an outline of `module_input` within a task configuration:
This `services` source input object defines services registered to Consul whose metadata will be used as [services source input to the Terraform Module](/docs/nia/terraform-modules/#services-source-input). The following parameters are supported:
```hcl
task {
name = "task_a"
module = "path/to/module"
services = ["api"] // (deprecated)
| Parameter | Required |Description | Default |
| --------- | --------- | ---------- | ------- |
| `regexp` | Optional | String value matching the names of Consul services to monitor. Only services that have a name matching the regular expression are used by the task. <br/><br/> If `regexp` is configured, then [`task.services`](#services) must be omitted or empty. <br/><br/> If both a list and a regex are needed, consider including the list as part of the regex or creating separate tasks. | none |
condition "<condition-type>" {
// ...
}
module_input "<input-type>" {
// ...
}
}
```
~> The type of the `module_input` block that can be configured depends on the `condition` block type and the `services` field (deprecated). See [Task Module Input Restrictions](/docs/nia/configuration#task-module-input-restrictions) for more details.
The following sections describe the module input types that CTS supports.
#### Services Module Input ((#services-source-input))
This `services` module input object defines services registered to Consul whose metadata will be used as [services module input to the Terraform Module](/docs/nia/terraform-modules/#services-module-input). The following parameters are supported:
| Parameter | Required | Type | Description | Default |
| --------- | -------- | ---- | ----------- | ------- |
| `regexp` | Required if `names` is not configured | string | Regular expression used to match the names of Consul services to monitor. Only services that have a name matching the regular expression are used by the task. <br/><br/> If both a list and a regex are needed, consider including the list as part of the regex or creating separate tasks. | none |
| `names` | Required if `regexp` is not configured | list[string] | Names of Consul services to monitor. Only services that have their name listed in `names` are used by the task. | none |
| `datacenter` | Optional | string | Name of a datacenter to query for the task. | Datacenter of the agent that CTS queries. |
| `namespace` | Optional | string |<EnterpriseAlert inline /> <br/> String value indicating the namespace of the services to query for the task. | In order of precedence: <br/> 1. Inferred from the CTS ACL token <br/> 2. The `default` namespace. |
| `filter` | Optional | string | Expression used to additionally filter the services to monitor. <br/><br/> Refer to the [services filtering documentation](/api-docs/health#filtering-2) and section about [how to write filter expressions](/api-docs/features/filtering#creating-expressions) for additional information. | none |
| `cts_user_defined_meta` | Optional | map[string] | User-defined metadata that is appended to the [service input variable](/docs/nia/terraform-modules#services-module-input) for compatible Terraform modules. <br/><br/> Some modules do not use the configured metadata. Refer to the module configured for the task for information about metadata usage and expected keys and format. | none|
In the following example, the scheduled task queries all Consul services with `web` as the suffix. The metadata of matching services are provided to the Terraform module.
@ -322,28 +411,34 @@ In the following example, the scheduled task queries all Consul services with `w
task {
name = "schedule_condition_task"
description = "execute every Monday using information from service names starting with web"
source = "path/to/module"
module = "path/to/module"
condition "schedule" {
cron = "* * * * Mon"
}
source_input “services” {
module_input “services” {
regexp = "^web.*"
datacenter = "dc1"
namespace = "default"
filter = "Service.Tags not contains \"prod\""
cts_user_defined_meta {
key = "value"
}
}
}
```
#### Consul KV Source Input
#### Consul KV Module Input ((#consul-kv-source-input))
A Consul KV source input block defines changes to Consul KV that will be monitored. These changes will then be provided as [Consul KV source input to the Terraform Module](/docs/nia/terraform-modules/#consul-kv-source-input). The source input can be configured for a single Consul KV entry or for any Consul KV entries that are prefixed with a given path. The following parameters are supported:
A Consul KV module input block defines changes to Consul KV that will be monitored. These changes will then be provided as [Consul KV module input to the Terraform Module](/docs/nia/terraform-modules/#consul-kv-module-input). The module input can be configured for a single Consul KV entry or for any Consul KV entries that are prefixed with a given path. The following parameters are supported:
| Parameter | Required | Description | Default |
| --------- | -------- | ----------- | ------- |
| `path` | Required | String value that specifies the path of the key used by the task. The path can point to a single Consul KV entry or several entries within the path. | none |
| `recurse` | Optional | Boolean value that enables Consul-Terraform-Sync to treat the path as a prefix. If set to `false`, the path will be treated as a literal match. | `false` |
| `datacenter` | Optional | String value specifying the name of a datacenter to query for the task. | Datacenter of the agent that Consul-Terraform-Sync queries. |
| `namespace` | Optional | <EnterpriseAlert inline /> <br/> String value indicating the namespace of the services to query for the task. | In order of precedence: <br/> 1. Inferred from the Consul-Terraform-Sync ACL token <br/> 2. The `default` namespace. |
| Parameter | Required | Type | Description | Default |
| --------- | -------- | ---- | ----------- | ------- |
| `path` | Required | string | Path of the key used by the task. The path can point to a single Consul KV entry or several entries within the path. | none |
| `recurse` | Optional | boolean | Enables CTS to treat the path as a prefix. If set to `false`, the path will be treated as a literal match. | `false` |
| `datacenter` | Optional | string | Name of a datacenter to query for the task. | Datacenter of the agent that CTS queries. |
| `namespace` | Optional | string | <EnterpriseAlert inline /> <br/> Namespace of the services to query for the task. | In order of precedence: <br/> 1. Inferred from the CTS ACL token <br/> 2. The `default` namespace. |
In the following example, the scheduled task queries datacenter `dc1` in the `default` namespace for changes to the value held by the key `my-key`.
@ -351,13 +446,13 @@ In the following example, the scheduled task queries datacenter `dc1` in the `de
task {
name = "schedule_condition_task_kv"
description = "execute every Monday using information from Consul KV entry my-key"
source = "path/to/module"
module = "path/to/module"
condition "schedule" {
cron = "* * * * Mon"
}
source_input "consul-kv" {
module_input "consul-kv" {
path = "my-key"
recurse = false
datacenter = "dc1"
@ -366,15 +461,28 @@ task {
}
```
#### Task Module Input Restrictions
There are some limitations to the type of `module_input` blocks that can be configured for a task given the task's `condition` block and `services` field (deprecated). This is because a task cannot have multiple configurations defining the same type of monitored variable:
- A task cannot be configured with a `condition` and `module_input` block of the same type. For example, configuring `condition "consul-kv"` and `module_input "consul-kv"` will error because both configure the `consul_kv` variable.
- A task cannot be configured with two or more `module_input` blocks of the same type. For example, configuring two `module_input "catalog-services"` within a task will return an error because they define multiple configurations for the `catalog_services` variable.
- A task that monitors services can only contain one of the following configurations:
- `condition "services"` block
- `module_input "services"` block
- Block was previously named `source_input "services"` (deprecated)
- `services` field (deprecated)
All of the listed configurations define the `services` variable and including more than one configuration will return an error.
## Network Drivers
A driver is required for Consul-Terraform-Sync to propagate network infrastructure change. The `driver` block configures the subprocess that Consul-Terraform-Sync runs in automation. The default driver is the [Terraform driver](#terraform-driver) which automates Terraform as a local installation of the Terraform CLI.
A driver is required for CTS to propagate network infrastructure change. The `driver` block configures the subprocess that CTS runs in automation. The default driver is the [Terraform driver](#terraform-driver) which automates Terraform as a local installation of the Terraform CLI.
Only one network driver can be configured per deployment of Consul-Terraform-Sync.
Only one network driver can be configured per deployment of CTS.
## Terraform Driver
The Terraform driver block is used to configure Consul-Terraform-Sync for installing and automating Terraform locally. The driver block supports Terraform configuration to specify the `backend` used for state management and `required_providers` configuration used for provider discovery.
The Terraform driver block is used to configure CTS for installing and automating Terraform locally. The driver block supports Terraform configuration to specify the `backend` used for state management and `required_providers` configuration used for provider discovery.
```hcl
driver "terraform" {
@ -395,16 +503,15 @@ driver "terraform" {
}
```
- `backend` - (obj) The backend stores [Terraform state files](https://www.terraform.io/docs/state/index.html) for each task. This option is similar to the [Terraform backend configuration](https://www.terraform.io/docs/configuration/backend.html). Consul-Terraform-Sync supports Terraform backends used as a state store.
- `backend` - (obj) The backend stores [Terraform state files](https://www.terraform.io/docs/state/index.html) for each task. This option is similar to the [Terraform backend configuration](https://www.terraform.io/docs/configuration/backend.html). CTS supports Terraform backends used as a state store.
- Supported backend options: [azurerm](https://www.terraform.io/docs/backends/types/azurerm.html), [consul](https://www.terraform.io/docs/backends/types/consul.html), [cos](https://www.terraform.io/docs/backends/types/cos.html), [gcs](https://www.terraform.io/docs/backends/types/gcs.html), [kubernetes](https://www.terraform.io/docs/backends/types/kubernetes.html), [local](https://www.terraform.io/docs/backends/types/local.html), [manta](https://www.terraform.io/docs/backends/types/manta.html), [pg](https://www.terraform.io/docs/backends/types/pg.html) (Terraform v0.14+), [s3](https://www.terraform.io/docs/backends/types/s3.html). Visit the Terraform documentation links for details on backend configuration options.
- If omitted, Consul-Terraform-Sync will generate default values and use configurations from the [`consul` block](#consul) to configure [Consul as the backend](https://www.terraform.io/docs/backends/types/consul.html), which stores Terraform statefiles in the Consul KV. The [ACL token provided for Consul authentication](#token) is used to read and write to the KV store and requires [Consul KV privileges](https://learn.hashicorp.com/tutorials/consul/consul-terraform-sync-secure?utm_source=WEBSITE&utm_medium=WEB_IO&utm_offer=ARTICLE_PAGE&utm_content=DOCS#configure-acl-privileges-for-consul-terraform-sync). The Consul KV path is the base path to store state files for tasks. The full path of each state file will have the task identifier appended to the end of the path, e.g. `consul-terraform-sync/terraform-env:task-name`.
- The remote enhanced backend is not supported with the Terraform driver to run operations in Terraform Cloud. Use the [Terraform Cloud driver](#terraform-cloud-driver) to integrate Consul-Terraform-Sync with Terraform Cloud for remote workspaces and remote operations.
- `log` - (bool) Enable all Terraform output (stderr and stdout) to be included in the Consul-Terraform-Sync log. This is useful for debugging and development purposes. It may be difficult to work with log aggregators that expect uniform log format.
- `path` - (string) The file path to install Terraform or discover an existing Terraform binary. If omitted, Terraform will be installed in the same directory as the Consul-Terraform-Sync daemon. To resolve an incompatible Terraform version or to change versions will require removing the existing binary or change to a different path.
- If omitted, CTS will generate default values and use configurations from the [`consul` block](#consul) to configure [Consul as the backend](https://www.terraform.io/docs/backends/types/consul.html), which stores Terraform statefiles in the Consul KV. The [ACL token provided for Consul authentication](#token) is used to read and write to the KV store and requires [Consul KV privileges](https://learn.hashicorp.com/tutorials/consul/consul-terraform-sync-secure?utm_source=WEBSITE&utm_medium=WEB_IO&utm_offer=ARTICLE_PAGE&utm_content=DOCS#configure-acl-privileges-for-consul-terraform-sync). The Consul KV path is the base path to store state files for tasks. The full path of each state file will have the task identifier appended to the end of the path, e.g. `consul-terraform-sync/terraform-env:task-name`.
- The remote enhanced backend is not supported with the Terraform driver to run operations in Terraform Cloud. Use the [Terraform Cloud driver](#terraform-cloud-driver) to integrate CTS with Terraform Cloud for remote workspaces and remote operations.
- `log` - (bool) Enable all Terraform output (stderr and stdout) to be included in the CTS log. This is useful for debugging and development purposes. It may be difficult to work with log aggregators that expect uniform log format.
- `path` - (string) The file path to install Terraform or discover an existing Terraform binary. If omitted, Terraform will be installed in the same directory as the CTS daemon. To resolve an incompatible Terraform version or to change versions will require removing the existing binary or change to a different path.
- `persist_log` - (bool) Enable trace logging for each Terraform client to disk per task. This is equivalent to setting `TF_LOG_PATH=<work_dir>/terraform.log`. Trace log level results in verbose logging and may be useful for debugging and development purposes. We do not recommend enabling this for production. There is no log rotation and may quickly result in large files.
- `required_providers` - (obj: required) Declare each Terraform provider used across all tasks. This can be configured the same as how you would configure [Terraform `terraform.required_providers`](https://www.terraform.io/docs/configuration/provider-requirements.html#requiring-providers) field to specify the source and version for each provider. Consul-Terraform-Sync will process these requirements when preparing each task that uses the provider.
- `version` - (string) The Terraform version to install and run in automation for task execution. If omitted, the driver will install the latest [compatible release of Terraform](/docs/nia/compatibility#terraform). To change versions, remove the existing binary or change the path to install the desired version. Verify that the desired Terraform version is compatible across all Terraform modules used for Consul-Terraform-Sync automation.
- `working_dir` - (string: "sync-tasks") **Deprecated in Consul-Terraform-Sync 0.3.0**, use the global [`working_dir`](#working_dir) option instead. The base working directory to manage Terraform configurations for all tasks. The full path of each working directory will have the task identifier appended to the end of the path, e.g. `./sync-tasks/task-name`.
- `required_providers` - (obj: required) Declare each Terraform provider used across all tasks. This can be configured the same as how you would configure [Terraform `terraform.required_providers`](https://www.terraform.io/docs/configuration/provider-requirements.html#requiring-providers) field to specify the source and version for each provider. CTS will process these requirements when preparing each task that uses the provider.
- `version` - (string) The Terraform version to install and run in automation for task execution. If omitted, the driver will install the latest [compatible release of Terraform](/docs/nia/compatibility#terraform). To change versions, remove the existing binary or change the path to install the desired version. Verify that the desired Terraform version is compatible across all Terraform modules used for CTS automation.
## Terraform Cloud Driver
@ -416,11 +523,11 @@ driver "terraform" {
which is available with <strong>Consul Enterprise</strong>.
</EnterpriseAlert>
The Terraform Cloud driver enables Consul-Terraform-Sync Enterprise to integrate with **Terraform Cloud**, including both the [self-hosted distribution](https://www.hashicorp.com/products/terraform/editions/enterprise) and the [managed service](https://www.hashicorp.com/products/terraform/editions/cloud). With this driver, Consul-Terraform-Sync automates Terraform runs and remote operations for workspaces.
The Terraform Cloud driver enables CTS Enterprise to integrate with **Terraform Cloud**, including both the [self-hosted distribution](https://www.hashicorp.com/products/terraform/editions/enterprise) and the [managed service](https://www.hashicorp.com/products/terraform/editions/cloud). With this driver, CTS automates Terraform runs and remote operations for workspaces.
An overview of features enabled with Terraform Cloud can be viewed within the [Network Drivers](/docs/nia/network-drivers) documentation.
Only one network driver can be configured per deployment of Consul-Terraform-Sync.
Only one network driver can be configured per deployment of CTS.
```hcl
driver "terraform-cloud" {
@ -447,15 +554,16 @@ driver "terraform-cloud" {
```
- `hostname` - (string) The Terraform Cloud hostname to connect to. Can be overridden with the `TFC_HOSTNAME` environment variable.
- `organization` - (string) The Terraform Cloud organization that hosts the managed workspaces by Consul-Terraform-Sync. Can be overridden with the `TFC_ORGANIZATION` environment variable.
- `token` - (string) Required [Team API token](https://www.terraform.io/docs/cloud/users-teams-organizations/api-tokens.html#team-api-tokens) used for authentication with Terraform Cloud and workspace management. Only workspace permissions are needed for Consul-Terraform-Sync. The token can also be provided using the `TFC_TOKEN` environment variable.
- We recommend creating a dedicated team and team API token to isolate automation by Consul-Terraform-Sync from other Terraform Cloud operations.
- `workspace_prefix` - (string) Specifies a prefix to prepend to the automatically-generated workspace names used for automation. This prefix will be used by all tasks that use this driver. By default, when no prefix is configured, the workspace name will be the task name. When a prefix is configured, the workspace name will be `<workspace_prefix value><task name>`. For example, if you configure the prefix as "cts_", then a task with the name "task_firewall" will have the workspace name "cts_task_firewall".
- `workspaces` - Configure Consul-Terraform-Sync management of Terraform Cloud workspaces.
- `organization` - (string) The Terraform Cloud organization that hosts the managed workspaces by CTS. Can be overridden with the `TFC_ORGANIZATION` environment variable.
- `token` - (string) Required [Team API token](https://www.terraform.io/docs/cloud/users-teams-organizations/api-tokens.html#team-api-tokens) used for authentication with Terraform Cloud and workspace management. Only workspace permissions are needed for CTS. The token can also be provided using the `TFC_TOKEN` environment variable.
- We recommend creating a dedicated team and team API token to isolate automation by CTS from other Terraform Cloud operations.
- `workspace_prefix` - (string) **Deprecated in CTS 0.5.0**, use the [`workspaces.prefix`](#prefix) option instead. Specifies a prefix to prepend to the automatically-generated workspace names used for automation. This prefix will be used by all tasks that use this driver. By default, when no prefix is configured, the workspace name will be the task name. When a prefix is configured, the workspace name will be `<workspace_prefix value>-<task name>`, with the character '-' between the workspace prefix and task name. For example, if you configure the prefix as "cts", then a task with the name "task-firewall" will have the workspace name "cts-task-firewall".
- `workspaces` - Configure CTS management of Terraform Cloud workspaces.
- `prefix` - (string) Specifies a prefix to prepend to the workspace names used for CTS task automation. This prefix will be used by all tasks that use this driver. By default, when no prefix is configured, the workspace name will be the task name. When a prefix is configured, the workspace name will be `<prefix><task name>`. For example, if you configure the prefix as "cts_", then a task with the name "task_firewall" will have the workspace name "cts_task_firewall".
- `tags` - (list[string]) Tags for CTS to add to all automated workspaces when the workspace is first created or discovered. Tags are added to discovered workspaces only if the workspace meets [automation requirements](/docs/nia/network-drivers/terraform-cloud#remote-workspaces) and satisfies the allowlist and denylist tag options. This option will not affect existing tags. Tags that were manually removed during runtime will be re-tagged when CTS restarts. Compatible with Terraform Cloud and Terraform Enterprise v202108-1+
- `tags_allowlist` - (list[string]) Tag requirement to use as a provision check for CTS automation of workspaces. When configured, Terraform Cloud workspaces must have at least one tag from the allow list for CTS to automate the workspace and runs. Compatible with Terraform Cloud and Terraform Enterprise v202108-1+.
- `tags_denylist` - (list[string]) Tag restriction to use as a provision check for CTS automation of workspaces. When configured, Terraform Cloud workspaces must not have any tag from the deny list for CTS to automate the workspace and runs. Denied tags have higher priority than tags set in the `tags_allowlist` option. Compatible with Terraform Cloud and Terraform Enterprise v202108-1+.
- `required_providers` - (obj: required) Declare each Terraform provider used across all tasks. This can be configured the same as how you would configure [Terraform `terraform.required_providers`](https://www.terraform.io/docs/configuration/provider-requirements.html#requiring-providers) field to specify the source and version for each provider. Consul-Terraform-Sync will process these requirements when preparing each task that uses the provider.
- `required_providers` - (obj: required) Declare each Terraform provider used across all tasks. This can be configured the same as how you would configure [Terraform `terraform.required_providers`](https://www.terraform.io/docs/configuration/provider-requirements.html#requiring-providers) field to specify the source and version for each provider. CTS will process these requirements when preparing each task that uses the provider.
- `tls` - Configure TLS to allow HTTPS connections to [Terraform Enterprise](https://www.terraform.io/docs/enterprise/install/installer.html#tls-key-amp-cert).
- `enabled` - (bool) Enable TLS. Providing a value for any of the TLS options will enable this parameter implicitly.
- `ca_cert` - (string) The path to a PEM-encoded certificate authority file used to verify the authenticity of the connection to Terraform Enterprise over TLS.
@ -471,7 +579,7 @@ driver "terraform-cloud" {
}
```
Consul-Terraform-Sync generates local artifacts to prepare configuration versions used for workspace runs. The location of the files created can be set with the [`working_dir`](/docs/nia/configuration#working_dir) option or configured per task. When a task is configured with a local module and is run with the Terraform Cloud driver, the local module is copied and uploaded as a part of the configuration version.
CTS generates local artifacts to prepare configuration versions used for workspace runs. The location of the files created can be set with the [`working_dir`](/docs/nia/configuration#working_dir) option or configured per task. When a task is configured with a local module and is run with the Terraform Cloud driver, the local module is copied and uploaded as a part of the configuration version.
The version of Terraform to use for each workspace can also be set within the [task](#task) configuration.
@ -497,30 +605,32 @@ terraform_provider "aws" {
}
task {
source = "some/source"
module = "path/to/module"
providers = ["aws"]
services = ["web", "api"]
condition "services" {
names = ["web", "api"]
}
}
```
~> **Note**: Provider arguments configured in Consul-Terraform-Sync configuration files are written in plain text to the generated [`terraform.tfvars`](/docs/nia/network-drivers#terraform-tfvars) file for each Terraform workspace that references the provider. To exclude arguments or dynamic values from rendering to local files in plain text, use [`task_env` in addition to using dynamic configuration](#securely-configure-terraform-providers).
~> **Note**: Provider arguments configured in CTS configuration files are written in plain text to the generated [`terraform.tfvars`](/docs/nia/network-drivers#terraform-tfvars) file for each Terraform workspace that references the provider. To exclude arguments or dynamic values from rendering to local files in plain text, use [`task_env` in addition to using dynamic configuration](#securely-configure-terraform-providers).
### Securely Configure Terraform Providers
The `terraform_provider` block supports dynamically loading arguments and the local environment from other sources. This can be used to securely configure your Terraform provider from the shell environment, Consul KV, or Vault. Using the `task_env` meta-argument and template syntax below, you can avoid exposing sensitive values or credentials in plain text within configuration files for Consul-Terraform-Sync.
The `terraform_provider` block supports dynamically loading arguments and the local environment from other sources. This can be used to securely configure your Terraform provider from the shell environment, Consul KV, or Vault. Using the `task_env` meta-argument and template syntax below, you can avoid exposing sensitive values or credentials in plain text within configuration files for CTS.
`task_env` and the template syntax for dynamic values are only supported within the `terraform_provider` block.
#### Provider Environment Variables
Terraform providers may support shell environment variables as values for some of their arguments. When available, we recommend using environment variables as a way to keep credentials out of plain-text configuration files. Refer to the official provider docs hosted on the [Terraform Registry](https://registry.terraform.io/browse/providers) to find supported environment variables for a provider. By default, Consul-Terraform-Sync enables all Terraform workspaces to inherit from its environment.
Terraform providers may support shell environment variables as values for some of their arguments. When available, we recommend using environment variables as a way to keep credentials out of plain-text configuration files. Refer to the official provider docs hosted on the [Terraform Registry](https://registry.terraform.io/browse/providers) to find supported environment variables for a provider. By default, CTS enables all Terraform workspaces to inherit from its environment.
The `task_env` block is a meta-argument available for the `terraform_provider` block that can be used to rename or scope the available environment to a selected set of variables. Passing sensitive values as environment variables will scope the values to only the tasks that require the provider.
```hcl
terraform_provider "foo" {
// Direct assignment of provider arguments are rendered in plain-text within
// the Consul-Terraform-Sync configuration and the generated terraform.tfvars
// the CTS configuration and the generated terraform.tfvars
// file for the corresponding Terraform workspaces.
// token = "<token value>"
@ -537,7 +647,7 @@ terraform_provider "foo" {
}
```
!> **Security note**: Consul-Terraform-Sync does not prevent sensitive values from being written to Terraform state files. We recommend securing state files in addition to securely configuring Terraform providers. Options for securing state files can be set within [`driver.backend`](#backend) based on the backend used. For example, Consul KV is the default backend and can be secured with ACLs for KV path. For other backends, we recommend enabling encryption, if applicable.
!> **Security note**: CTS does not prevent sensitive values from being written to Terraform state files. We recommend securing state files in addition to securely configuring Terraform providers. Options for securing state files can be set within [`driver.backend`](#backend) based on the backend used. For example, Consul KV is the default backend and can be secured with ACLs for KV path. For other backends, we recommend enabling encryption, if applicable.
#### Load Dynamic Values
@ -545,7 +655,7 @@ Load dynamic values for Terraform providers with integrated template syntax.
##### Env
`env` reads the given environment variable accessible to Consul-Terraform-Sync.
`env` reads the given environment variable accessible to CTS.
```hcl
terraform_provider "example" {
@ -591,7 +701,7 @@ terraform_provider "example" {
- `VAULT_SKIP_VERIFY`
- `VAULT_TLS_SERVER_NAME`
- `token` - (string) Token is the Vault token to communicate with for requests. It may be a wrapped token or a real token. This can also be set via the `VAULT_TOKEN` environment variable, or via the `VaultAgentTokenFile`.
- `vault_agent_token_file` - (string) The path of the file that contains a Vault Agent token. If this is specified, Consul-Terraform-Sync will not try to renew the Vault token.
- `vault_agent_token_file` - (string) The path of the file that contains a Vault Agent token. If this is specified, CTS will not try to renew the Vault token.
- `transport` - [(transport block)](#transport) Transport configures the low-level network connection details.
- `unwrap_token` - (bool) Unwraps the provided Vault token as a wrapped token.
@ -599,9 +709,9 @@ terraform_provider "example" {
### Multiple Provider Configurations
Consul-Terraform-Sync supports the [Terraform feature to define multiple configurations](https://www.terraform.io/docs/configuration/providers.html#alias-multiple-provider-configurations) for the same provider by utilizing the `alias` meta-argument. Define multiple provider blocks with the same provider name and set the `alias` to a unique value across a given provider. Select which provider configuration to use for a task by specifying the configuration with the provider name and alias (`<name>.<alias>`) within the list of providers in the [`task.provider`](#task) parameter. A task can use multiple providers, but only one provider instance of a provider is allowed per task.
CTS supports the [Terraform feature to define multiple configurations](https://www.terraform.io/docs/configuration/providers.html#alias-multiple-provider-configurations) for the same provider by utilizing the `alias` meta-argument. Define multiple provider blocks with the same provider name and set the `alias` to a unique value across a given provider. Select which provider configuration to use for a task by specifying the configuration with the provider name and alias (`<name>.<alias>`) within the list of providers in the [`task.provider`](#task) parameter. A task can use multiple providers, but only one provider instance of a provider is allowed per task.
The example Consul-Terraform-Sync configuration below defines two similar tasks executing the same module with different instances of the AWS provider.
The example CTS configuration below defines two similar tasks executing the same module with different instances of the AWS provider.
```hcl
terraform_provider "aws" {
@ -626,14 +736,14 @@ terraform_provider "dns" {
task {
name = "task-a"
source = "org/module"
module = "org/module"
providers = ["aws.a", "dns"]
// ...
}
task {
name = "task-b"
source = "org/module"
module = "org/module"
providers = ["aws.b", "dns"]
// ...
}

18
website/content/docs/nia/enterprise/license.mdx
View File

@ -8,14 +8,14 @@ description: >-
# Consul-Terraform-Sync Enterprise License
<EnterpriseAlert>
Licenses are only required for Consul-Terraform-Sync Enterprise
Licenses are only required for Consul-Terraform-Sync (CTS) Enterprise
</EnterpriseAlert>
Consul-Terraform-Sync Enterprise binaries require a [Consul Enterprise license](/docs/enterprise/license/overview) to run. There is no Consul-Terraform-Sync Enterprise specific license. As a result, Consul-Terraform-Sync Enterprise's licensing is very similar to Consul Enterprise.
CTS Enterprise binaries require a [Consul Enterprise license](/docs/enterprise/license/overview) to run. There is no CTS Enterprise specific license. As a result, CTS Enterprise's licensing is very similar to Consul Enterprise.
All Consul-Terraform-Sync Enterprise features are available with a valid Consul Enterprise license, regardless of your Consul Enterprise packaging or pricing model.
All CTS Enterprise features are available with a valid Consul Enterprise license, regardless of your Consul Enterprise packaging or pricing model.
To get a trial license for Consul-Terraform-Sync Enterprise, you can sign-up for the [trial license for Consul Enterprise](/docs/enterprise/license/faq#q-where-can-users-get-a-trial-license-for-consul-enterprise).
To get a trial license for CTS, you can sign-up for the [trial license for Consul Enterprise](/docs/enterprise/license/faq#q-where-can-users-get-a-trial-license-for-consul-enterprise).
## Setting the License
@ -44,20 +44,20 @@ Visit the [Enterprise License Tutorial](https://learn.hashicorp.com/tutorials/no
### Updating the License
The previous section describes options for [setting the license](#setting-the-license) needed to start running Consul-Terraform-Sync Enterprise. Use the following procedure to update the license when it expires or is near the expiration date:
The previous section describes options for [setting the license](#setting-the-license) needed to start running CTS Enterprise. Use the following procedure to update the license when it expires or is near the expiration date:
1. Update the license environment variable or configuration with the new license value or path to the new license file
1. Stop and restart Consul-Terraform-Sync Enterprise
1. Stop and restart CTS Enterprise
Once Consul-Terraform-Sync Enterprise starts again, it will pick up the new license and run the tasks with any changes that may have occurred between the stop and restart period.
Once CTS Enterprise starts again, it will pick up the new license and run the tasks with any changes that may have occurred between the stop and restart period.
## Notification and Termination Behavior
Licenses have an expiration date and a termination date. The termination date is a time at or after the license expires. Consul-Terraform-Sync Enterprise will cease to function once the termination date has passed.
Licenses have an expiration date and a termination date. The termination date is a time at or after the license expires. CTS Enterprise will cease to function once the termination date has passed.
The time between the expiration and termination dates is a grace period. Grace periods are generally 24-hours, but you should refer to your license agreement for complete terms of your grace period.
When approaching expiration and termination, Consul-Terraform-Sync Enterprise will provide notifications in the system logs:
When approaching expiration and termination, CTS Enterprise will provide notifications in the system logs:
| Time period | Behavior |
| ------------------------------------------- | --------------------------------- |

54
website/content/docs/nia/index.mdx
View File

@ -7,55 +7,65 @@ description: >-
# Network Infrastructure Automation
Network Infrastructure Automation (NIA) enables dynamic updates to network infrastructure devices triggered by service changes. Consul-Terraform-Sync utilizes Consul as a data source that contains networking information about services and monitors those services. Terraform is used as the underlying automation tool and leverages the Terraform provider ecosystem to drive relevant changes to the network infrastructure.
Network Infrastructure Automation (NIA) enables dynamic updates to network infrastructure devices triggered by service changes. Consul-Terraform-Sync (CTS) utilizes Consul as a data source that contains networking information about services and monitors those services. Terraform is used as the underlying automation tool and leverages the Terraform provider ecosystem to drive relevant changes to the network infrastructure.
Consul-Terraform-Sync executes one or more automation tasks with the most recent service variable values from the Consul service catalog. Each task consists of a runbook automation written as a Consul-Terraform-Sync compatible Terraform module using resources and data sources for the underlying network infrastructure. The `consul-terraform-sync` daemon runs on the same node as a Consul agent.
CTS executes one or more automation tasks with the most recent service variable values from the Consul service catalog. Each task consists of a runbook automation written as a CTS compatible Terraform module using resources and data sources for the underlying network infrastructure. The `consul-terraform-sync` daemon runs on the same node as a Consul agent.
CTS is available as an open source and enterprise distribution. Follow the [Network Infrastructure Automation introduction tutorial](https://learn.hashicorp.com/tutorials/consul/consul-terraform-sync-intro?utm_source=WEBSITE&utm_medium=WEB_IO&utm_offer=ARTICLE_PAGE&utm_content=DOCS) to get started with CTS OSS or read more about [CTS Enterprise](/docs/nia/enterprise).
## Use Cases
**Application teams must wait for manual changes in the network to release, scale up/down and re-deploy their applications.** This creates a bottleneck, especially in frequent workflows related to scaling up/down the application, breaking the DevOps goal of self-service enablement. Consul-Terraform-Sync automates this process, thus decreasing the possibility of human error in manually editing configuration files, as well as decreasing the overall time taken to push out configuration changes.
**Application teams must wait for manual changes in the network to release, scale up/down and re-deploy their applications.** This creates a bottleneck, especially in frequent workflows related to scaling up/down the application, breaking the DevOps goal of self-service enablement. CTS automates this process, thus decreasing the possibility of human error in manually editing configuration files, as well as decreasing the overall time taken to push out configuration changes.
**Networking and security teams cannot scale processes to the speed and changes needed.** Manual approaches don't scale well, causing backlogs in network and security teams. Even in organizations that have some amount of automation (such as scripting), there is a need for an accurate, real-time source of data to trigger and drive their network automation workflows. Consul-Terraform-Sync runs in near real-time to keep up with the rate of change.
**Networking and security teams cannot scale processes to the speed and changes needed.** Manual approaches don't scale well, causing backlogs in network and security teams. Even in organizations that have some amount of automation (such as scripting), there is a need for an accurate, real-time source of data to trigger and drive their network automation workflows. CTS runs in near real-time to keep up with the rate of change.
## Glossary
**Condition** - A task-level defined environmental requirement. When a task’s condition is met, Consul-Terraform-Sync executes that task to update network infrastructure. Depending on the condition type, the condition definition may also define and enable the source input that the task provides to the configured Terraform module.
- `Condition` - A task-level defined environmental requirement. When a task's condition is met, CTS executes that task to update network infrastructure. Depending on the condition type, the condition definition may also define and enable the module input that the task provides to the configured Terraform Module.
**Consul-Terraform-Sync** - [GitHub repo](https://github.com/hashicorp/consul-terraform-sync) and binary/CLI name for the project that is used to perform Network Infrastructure Automation.
- `Consul objects` - Consul objects are the response request objects returned from the Consul API that CTS monitor for changes. Examples of Consul objects can be service instance information, Consul key-value pairs, and service registration. The Consul objects are used to inform a task's condition and/or module input.
**Dynamic Tasks** - A dynamic task is a type of task that is dynamically triggered on a change to any relevant Consul catalog values e.g. service instances, Consul KV, catalog-services. See scheduled tasks for a type of non-dynamic task.
- `Consul-Terraform-Sync (CTS)` - [GitHub repo](https://github.com/hashicorp/consul-terraform-sync) and binary/CLI name for the project that is used to perform Network Infrastructure Automation.
-> **Note:** The terminology "tasks" used throughout the documentation refers to all types of tasks except when specifically stated otherwise.
- `Dynamic Tasks` - A dynamic task is a type of task that is dynamically triggered on a change to any relevant Consul catalog values e.g. service instances, Consul KV, catalog-services. See scheduled tasks for a type of non-dynamic task.
**Network Drivers** - Consul-Terraform-Sync uses [network drivers](/docs/nia/network-drivers) to execute and update network infrastructure. Drivers transform Consul service-level information into downstream changes by processing and abstracting API and resource details tied to specific network infrastructure.
-> **Note:** The terminology "tasks" used throughout the documentation refers to all types of tasks except when specifically stated otherwise.
**Scheduled Tasks** - A scheduled task is a type of task that is triggered only on a schedule. It is configured with a [schedule condition](/docs/nia/configuration#schedule-condition).
- `Network Drivers` - CTS uses [network drivers](/docs/nia/network-drivers) to execute and update network infrastructure. Drivers transform Consul service-level information into downstream changes by processing and abstracting API and resource details tied to specific network infrastructure.
-> **Note:** The terminology "tasks" used throughout the documentation refers to all types of tasks except when specifically stated otherwise.
- `Network Infrastructure Automation (NIA)` - Enables dynamic updates to network infrastructure devices triggered when specific conditions, such as service changes and registration, are met.
**Source Input** - A source input defines objects that provide values or metadata to the Terraform module. See [source input](/docs/nia/terraform-modules#source-input) for the supported metadata and values. For example, a user can configure a Consul KV source input to provide KV pairs as variables to their respective Terraform module. The source input can be included in two ways. It can be specified as a parameter in a condition using `source_includes_var` and also by using the `source_input` block.
- `Scheduled Tasks` - A scheduled task is a type of task that is triggered only on a schedule. It is configured with a [schedule condition](/docs/nia/configuration#schedule-condition).
**Network Infrastructure Automation (NIA)** - Enables dynamic updates to network infrastructure devices triggered when specific conditions, such as service changes and registration, are met.
- `Services` - A service in CTS represents a service that is registered with Consul for service discovery. Services are grouped by their service names. There may be more than one instance of a particular service, each with its own unique ID. CTS monitors services based on service names and can provide service instance details to a Terraform module for network automation.
**Services** - A service in CTS represents a service that is registered with Consul for service discovery. Services are grouped by their service names. There may be more than one instance of a particular service, each with its own unique ID. CTS monitors services based on service names and can provide service instance details to a Terraform module for network automation.
- `Module Input` - A module input defines objects that provide values or metadata to the Terraform module. See [module input](/docs/nia/terraform-modules#module-input) for the supported metadata and values. For example, a user can configure a Consul KV module input to provide KV pairs as variables to their respective Terraform Module.
**Tasks** - A task is the translation of dynamic service information from the Consul Catalog into network infrastructure changes downstream.
The module input can be configured in a couple of ways:
- Setting the `condition` block's `use_as_module_input` field to true
- Field was previously named `source_includes_var` (deprecated)
- Configuring `module_input` block(s)
- Block was previously named `source_input` (deprecated)
**Terraform Cloud** - Per the [Terraform documentation](https://www.terraform.io/docs/cloud/index.html), "Terraform Cloud" describes both Terraform Cloud and Terraform Enterprise, which are different distributions of the same application. Documentation will apply to both distributions unless specifically stated otherwise.
~> "Module input" was renamed from "source input" in CTS 0.5.0 due to updates to the configuration names seen above.
**Terraform Module** - A [Terraform module](https://www.terraform.io/docs/configuration/modules.html) is a container for multiple Terraform resources that are used together.
-> **Note:** The terminology "tasks" used throughout the documentation refers to all types of tasks except when specifically stated otherwise.
**Terraform Provider** - A [Terraform provider](https://www.terraform.io/docs/providers/index.html) is responsible for understanding API interactions and exposing resources for an infrastructure type.
- `Tasks` - A task is the translation of dynamic service information from the Consul Catalog into network infrastructure changes downstream.
- `Terraform Cloud` - Per the [Terraform documentation](https://www.terraform.io/docs/cloud/index.html), "Terraform Cloud" describes both Terraform Cloud and Terraform Enterprise, which are different distributions of the same application. Documentation will apply to both distributions unless specifically stated otherwise.
- `Terraform Module` - A [Terraform module](https://www.terraform.io/docs/configuration/modules.html) is a container for multiple Terraform resources that are used together.
- `Terraform Provider` - A [Terraform provider](https://www.terraform.io/docs/providers/index.html) is responsible for understanding API interactions and exposing resources for an infrastructure type.
## Getting Started With Network Infrastructure Automation
The [Network Infrastructure Automation (NIA)](https://learn.hashicorp.com/collections/consul/network-infrastructure-automation?utm_source=WEBSITE&utm_medium=WEB_IO&utm_offer=ARTICLE_PAGE&utm_content=DOCS)
collection contains examples on how to configure Consul-Terraform-Sync to
collection contains examples on how to configure CTS to
perform Network Infrastructure Automation. The collection contains also a
tutorial to secure your Consul-Terraform-Sync configuration for a production
environment and one to help you build you own Consul-Terraform-Sync compatible
tutorial to secure your CTS configuration for a production
environment and one to help you build you own CTS compatible
module.
## Community
@ -63,4 +73,4 @@ module.
- [Contribute](https://github.com/hashicorp/consul-terraform-sync) to the open source project
- [Report](https://github.com/hashicorp/consul-terraform-sync/issues) bugs or request enhancements
- [Discuss](https://discuss.hashicorp.com/tags/c/consul/29/consul-terraform-sync) with the community or ask questions
- [Build integrations](/docs/nia/terraform-modules) for Consul-Terraform-Sync
- [Build integrations](/docs/nia/terraform-modules) for CTS

24
website/content/docs/nia/installation/configure.mdx
View File

@ -7,13 +7,13 @@ description: >-
# Configure Consul-Terraform-Sync
The page will cover the main components for configuring your Network Infrastructure Automation with Consul at a high level. For the full list of configuration options, visit the [Consul-Terraform-Sync Configuration page](/docs/nia/configuration).
The page will cover the main components for configuring your Network Infrastructure Automation with Consul at a high level. For the full list of configuration options, visit the [Consul-Terraform-Sync (CTS) configuration page](/docs/nia/configuration).
## Tasks
A task captures a network automation process by defining which network resources to update on a given condition. Configure Consul-Terraform-Sync with one or more tasks that contain a list of Consul services, a Terraform module, and various Terraform providers.
A task captures a network automation process by defining which network resources to update on a given condition. Configure CTS with one or more tasks that contain a list of Consul services, a Terraform module, and various Terraform providers.
Within the [`task` block](/docs/nia/configuration#task), the list of services for a task represents the service layer that drives network automation. The `source` is the discovery location of the Terraform module that defines the network automation process for the task. The `condition`, not shown below, defaults to the services condition when unconfigured such that network resources are updated on changes to the list of services over time.
Within the [`task` block](/docs/nia/configuration#task), the list of services for a task represents the service layer that drives network automation. The `module` is the discovery location of the Terraform module that defines the network automation process for the task. The `condition`, not shown below, defaults to the services condition when unconfigured such that network resources are updated on changes to the list of services over time.
Review the Terraform module to be used for network automation and identify the Terraform providers required by the module. If the module depends on a set of providers, include the list of provider names in the `providers` field to associate the corresponding provider configuration with the task. These providers will need to be configured later in a separate block.
@ -21,16 +21,18 @@ Review the Terraform module to be used for network automation and identify the T
task {
name = "website-x"
description = "automate services for website-x"
source = "namespace/example/module"
module = "namespace/example/module"
version = "1.0.0"
providers = ["myprovider"]
services = ["web", "api"]
condition "services" {
names = ["web", "api"]
}
}
```
## Terraform Providers
Configuring Terraform providers within Consul-Terraform-Sync requires 2 config components. The first component is required within the [`driver.terraform` block](/docs/nia/configuration#terraform-driver). All providers configured for Consul-Terraform-Sync must be listed within the `required_providers` stanza to satisfy a [Terraform v0.13+ requirement](https://www.terraform.io/docs/configuration/provider-requirements.html#requiring-providers) for Terraform to discover and install them. The providers listed are later organized by Consul-Terraform-Sync to be included in the appropriate Terraform configuration files for each task.
Configuring Terraform providers within CTS requires 2 config components. The first component is required within the [`driver.terraform` block](/docs/nia/configuration#terraform-driver). All providers configured for CTS must be listed within the `required_providers` stanza to satisfy a [Terraform v0.13+ requirement](https://www.terraform.io/docs/configuration/provider-requirements.html#requiring-providers) for Terraform to discover and install them. The providers listed are later organized by CTS to be included in the appropriate Terraform configuration files for each task.
```hcl
driver "terraform" {
@ -45,7 +47,7 @@ driver "terraform" {
The second component for configuring a provider is the [`terraform_provider` block](/docs/nia/configuration#terraform-provider). This block resembles [provider blocks for Terraform configuration](https://www.terraform.io/docs/configuration/providers.html) and has the same responsibility for understanding API interactions and exposing resources for a specific infrastructure platform.
Terraform modules configured for task automation may require configuring the referenced providers. For example, configuring the host address and authentication to interface with your network infrastructure. Refer to the Terraform provider documentation hosted on the [Terraform Registry](https://registry.terraform.io/browse/providers) to find available options. The `terraform_provider` block is loaded by Consul-Terraform-Sync during runtime and processed to be included in [autogenerated Terraform configuration files](/docs/nia/network-drivers#provider) used for task automation. Omitting the `terraform_provider` block for a provider will defer to the Terraform behavior assuming an empty default configuration.
Terraform modules configured for task automation may require configuring the referenced providers. For example, configuring the host address and authentication to interface with your network infrastructure. Refer to the Terraform provider documentation hosted on the [Terraform Registry](https://registry.terraform.io/browse/providers) to find available options. The `terraform_provider` block is loaded by CTS during runtime and processed to be included in [autogenerated Terraform configuration files](/docs/nia/network-drivers#provider) used for task automation. Omitting the `terraform_provider` block for a provider will defer to the Terraform behavior assuming an empty default configuration.
```hcl
terraform_provider "myprovider" {
@ -55,7 +57,7 @@ terraform_provider "myprovider" {
## Summary
Piecing it all together, the configuration file for Consul-Terraform-Sync will have several HCL blocks in addition to other options for configuring the Consul-Terraform-Sync daemon: `task`, `driver.terraform`, and `terraform_provider` blocks.
Piecing it all together, the configuration file for CTS will have several HCL blocks in addition to other options for configuring the CTS daemon: `task`, `driver.terraform`, and `terraform_provider` blocks.
An example HCL configuration file is shown below to automate one task to execute a Terraform module on the condition when there are changes to two services.
@ -75,10 +77,12 @@ consul {
task {
name = "website-x"
description = "automate services for website-x"
source = "namespace/example/module"
module = "namespace/example/module"
version = "1.0.0"
providers = ["myprovider"]
services = ["web", "api"]
condition "services" {
names = ["web", "api"]
}
buffer_period {
min = "10s"
}

12
website/content/docs/nia/installation/install.mdx
View File

@ -7,14 +7,14 @@ description: >-
# Install Consul-Terraform-Sync
Refer to the [introduction](https://learn.hashicorp.com/tutorials/consul/consul-terraform-sync-intro?utm_source=WEBSITE&utm_medium=WEB_IO&utm_offer=ARTICLE_PAGE&utm_content=DOCS) tutorial for details about installing, configuring, and running Consul-Terraform-Sync on your local machine with the Terraform driver.
Refer to the [introduction](https://learn.hashicorp.com/tutorials/consul/consul-terraform-sync-intro?utm_source=WEBSITE&utm_medium=WEB_IO&utm_offer=ARTICLE_PAGE&utm_content=DOCS) tutorial for details about installing, configuring, and running Consul-Terraform-Sync (CTS) on your local machine with the Terraform driver.
## Install Consul-Terraform-Sync
<Tabs>
<Tab heading="Pre-compiled binary">
To install Consul-Terraform-Sync, find the [appropriate package](https://releases.hashicorp.com/consul-terraform-sync/) for your system and download it as a zip archive. For the CTS Enterprise binary, download a zip archive with the `+ent` metadata. [CTS Enterprise requires a Consul Enterprise license](/docs/nia/enterprise/license) to run.
To install CTS, find the [appropriate package](https://releases.hashicorp.com/consul-terraform-sync/) for your system and download it as a zip archive. For the CTS Enterprise binary, download a zip archive with the `+ent` metadata. [CTS Enterprise requires a Consul Enterprise license](/docs/nia/enterprise/license) to run.
Unzip the package to extract the binary named `consul-terraform-sync`. Move the `consul-terraform-sync` binary to a location available on your `PATH`.
@ -35,7 +35,7 @@ $ consul-terraform-sync -version
</Tab>
<Tab heading="Docker">
Install and run Consul-Terraform-Sync as a [Docker container](https://hub.docker.com/r/hashicorp/consul-terraform-sync).
Install and run CTS as a [Docker container](https://hub.docker.com/r/hashicorp/consul-terraform-sync).
For the CTS Enterprise, use the Docker image [`hashicorp/consul-terraform-sync-enterprise`](https://hub.docker.com/r/hashicorp/consul-terraform-sync-enterprise).
@ -94,7 +94,7 @@ $ consul-terraform-sync -version
## Connect your Consul Cluster
Consul-Terraform-Sync connects with your Consul cluster in order to monitor the Consul catalog for service changes. These service changes lead to downstream updates to your network devices. You can configure your Consul cluster in Consul-Terraform-Sync with the [Consul block](/docs/nia/configuration#consul). Below is an example:
CTS connects with your Consul cluster in order to monitor the Consul catalog for service changes. These service changes lead to downstream updates to your network devices. You can configure your Consul cluster in CTS with the [Consul block](/docs/nia/configuration#consul). Below is an example:
```hcl
consul {
@ -105,9 +105,9 @@ consul {
## Connect your Network Device
Consul-Terraform-Sync interacts with your network device through a network driver. For the Terraform network driver, Consul-Terraform-Sync uses Terraform providers to make changes to your network infrastructure resources. You can reference existing provider docs on the Terraform Registry to configure each provider or create a new Terraform provider.
CTS interacts with your network device through a network driver. For the Terraform network driver, CTS uses Terraform providers to make changes to your network infrastructure resources. You can reference existing provider docs on the Terraform Registry to configure each provider or create a new Terraform provider.
Once you have identified a Terraform provider for all of your network devices, you can configure them in Consul-Terraform-Sync with a [`terraform_provider` block](/docs/nia/configuration#terraform-provider) for each network device. Below is an example:
Once you have identified a Terraform provider for all of your network devices, you can configure them in CTS with a [`terraform_provider` block](/docs/nia/configuration#terraform-provider) for each network device. Below is an example:
```hcl
terraform_provider "fake-firewall" {

32
website/content/docs/nia/installation/requirements.mdx
View File

@ -2,28 +2,28 @@
layout: docs
page_title: Requirements
description: >-
Consul-Terraform-Sync requires a Terraform Provider, a Terraform Module, and a running Consul cluster outside of the consul-terraform-sync daemon.
Consul-Terraform-Sync requires a Terraform Provider, a Terraform Module, and a running Consul cluster outside of the `consul-terraform-sync` daemon.
---
# Prerequisites Needed to Run Consul-Terraform-Sync
The following components are required to run Consul-Terraform-Sync:
The following components are required to run Consul-Terraform-Sync (CTS):
* A Terraform Provider
* A Terraform Module
* A Consul cluster running outside of the consul-terraform-sync daemon
* A Consul cluster running outside of the `consul-terraform-sync` daemon
Practitioners can add support for their network infrastructure through Terraform providers. Once network infrastructure support exists, practitioners can add network integrations in the form of Terraform modules.
The following guidance is for running Consul-Terraform-Sync using the Terraform driver. The Terraform Cloud driver<EnterpriseAlert inline />has [additional prerequisites](/docs/nia/network-drivers/terraform-cloud#setting-up-terraform-cloud-driver).
The following guidance is for running CTS using the Terraform driver. The Terraform Cloud driver<EnterpriseAlert inline />has [additional prerequisites](/docs/nia/network-drivers/terraform-cloud#setting-up-terraform-cloud-driver).
## Run a Consul Cluster
Below are several steps towards a minimum Consul setup required for running Consul-Terraform-Sync.
Below are several steps towards a minimum Consul setup required for running CTS.
### Install Consul
Consul-Terraform-Sync is a daemon that runs alongside Consul, similar to other Consul ecosystem tools like Consul Template. Consul-Terraform-Sync is not included with the Consul binary and needs to be installed separately.
CTS is a daemon that runs alongside Consul, similar to other Consul ecosystem tools like Consul Template. CTS is not included with the Consul binary and needs to be installed separately.
To install a local Consul agent, refer to the [Getting Started: Install Consul Tutorial](https://learn.hashicorp.com/tutorials/consul/get-started-install).
@ -33,13 +33,13 @@ For information on compatible Consul versions, refer to the [Consul compatibilit
The Consul agent must be running in order to dynamically update network devices. To run the local Consul agent, you can run Consul in development mode which can be started with `consul agent -dev` for simplicity. For more details on running Consul agent, refer to the [Getting Started: Run the Consul Agent Tutorial](https://learn.hashicorp.com/tutorials/consul/get-started-agent?in=consul/getting-started).
When running a Consul agent with Consul-Terraform-Sync in production, we suggest to keep a few considerations in mind. Consul-Terraform-Sync uses [blocking queries](/api/features/blocking) to monitor task dependencies, like changes to registered services. This results in multiple long running TCP connections between Consul-Terraform-Sync and the agent to poll changes for each dependency. Monitoring a high number of services may quickly hit the default Consul agent connection limits.
When running a Consul agent with CTS in production, we suggest to keep a few considerations in mind. CTS uses [blocking queries](/api/features/blocking) to monitor task dependencies, like changes to registered services. This results in multiple long running TCP connections between CTS and the agent to poll changes for each dependency. Monitoring a high number of services may quickly hit the default Consul agent connection limits.
There are 2 ways to fix this issue. The first and recommended fix is to use HTTP/2 (requires HTTPS) to communicate between Consul-Terraform-Sync and the Consul agent. When using HTTP/2 only a single connection is made and reused for all communications. See the [Consul Configuration section](/docs/nia/configuration#consul) for more. The other option is to configure [`limits.http_max_conns_per_client`](/docs/agent/options#http_max_conns_per_client) for the agent to a reasonable value proportional to the number of services monitored by Consul-Terraform-Sync.
There are 2 ways to fix this issue. The first and recommended fix is to use HTTP/2 (requires HTTPS) to communicate between CTS and the Consul agent. When using HTTP/2 only a single connection is made and reused for all communications. See the [Consul Configuration section](/docs/nia/configuration#consul) for more. The other option is to configure [`limits.http_max_conns_per_client`](/docs/agent/options#http_max_conns_per_client) for the agent to a reasonable value proportional to the number of services monitored by CTS.
### Register Services
Consul-Terraform-Sync monitors Consul catalog for service changes which lead to downstream changes to your network devices. Without services, your Consul-Terraform-Sync daemon will be operational but idle. You can register services with your Consul agent either by loading a service definition or by HTTP API request.
CTS monitors Consul catalog for service changes which lead to downstream changes to your network devices. Without services, your CTS daemon will be operational but idle. You can register services with your Consul agent either by loading a service definition or by HTTP API request.
If you are running Consul in development mode, below is an example of registering a service by HTTP API request:
@ -54,7 +54,7 @@ $ echo '{
$ curl --request PUT --data @payload.json http://localhost:8500/v1/agent/service/register
```
The above example registers a service named "web" with your Consul agent. This represents a non-existent web service running at 10.10.10.10:8000. Your web service is now available for Consul-Terraform-Sync to consume. In Consul-Terraform-Sync, you can optionally configure the web service with a [service block](/docs/nia/configuration#service) if it has any non-default values. You can also have Consul-Terraform-Sync monitor the web service to execute a task and update network device(s) by configuring "web" in [`task.services`](/docs/nia/configuration#services) of a task block.
The above example registers a service named "web" with your Consul agent. This represents a non-existent web service running at 10.10.10.10:8000. Your web service is now available for CTS to consume. You can have CTS monitor the web service to execute a task and update network device(s) by configuring "web" in [`condition "services"`](/docs/nia/configuration#services-condition) of a task block. If the web service has any non-default values, it can also be configured in `condition "services"`.
For more details on registering a service by HTTP API request, refer to the [register service API docs](/api-docs/agent/service#register-service).
@ -62,13 +62,13 @@ For more details on registering a service by loading a service definition, refer
### Run a Cluster
The previous steps of installing and running a single Consul agent then registering a single service is sufficient to meaningfully start running Consul-Terraform-Sync.
The previous steps of installing and running a single Consul agent then registering a single service is sufficient to meaningfully start running CTS.
If you would like to run a Consul cluster rather than a single agent, refer to [Getting Started: Create a Local Consul Datacenter](https://learn.hashicorp.com/tutorials/consul/get-started-create-datacenter?in=consul/getting-started). This will walk you through the steps of running multiple Consul agents and then joining them together into a cluster.
## Network Infrastructure (using a Terraform Provider)
Consul-Terraform-Sync integrations for the Terraform driver utilizes Terraform providers as plugins to interface with specific network infrastructure platforms. The Terraform driver of Consul-Terraform-Sync inherits the expansive collection of Terraform providers to integrate with, and with release of [Terraform 0.13](https://www.hashicorp.com/blog/announcing-hashicorp-terraform-0-13/), this extends to include providers written by the community too by using [provider source](https://www.hashicorp.com/blog/adding-provider-source-to-hashicorp-terraform/).
CTS integrations for the Terraform driver utilizes Terraform providers as plugins to interface with specific network infrastructure platforms. The Terraform driver of CTS inherits the expansive collection of Terraform providers to integrate with, and with release of [Terraform 0.13](https://www.hashicorp.com/blog/announcing-hashicorp-terraform-0-13/), this extends to include providers written by the community too by using [provider source](https://www.hashicorp.com/blog/adding-provider-source-to-hashicorp-terraform/).
### Finding Terraform Providers
@ -80,15 +80,15 @@ If there is no existing Terraform provider, a new Terraform provider can be [cre
## Network Integration (using a Terraform Module)
The Terraform module for a task in Consul-Terraform-Sync is the core component of the integration. It declares which resources and how your infrastructure is dynamically updated. The module along with how it is configured within a task determines the condition under which your infrastructure is updated.
The Terraform module for a task in CTS is the core component of the integration. It declares which resources and how your infrastructure is dynamically updated. The module along with how it is configured within a task determines the condition under which your infrastructure is updated.
Working with a Terraform provider, you can write an integration task for Consul-Terraform-Sync by [creating a Terraform module](/docs/nia/terraform-modules) that is compatible with the Terraform driver or use a module built by partners below.
Working with a Terraform provider, you can write an integration task for CTS by [creating a Terraform module](/docs/nia/terraform-modules) that is compatible with the Terraform driver or use a module built by partners below.
Continue to the next page to [get started with configuring Consul-Terraform-Sync and how to use Terraform providers and modules for tasks.](/docs/nia/installation/configure)
Continue to the next page to [get started with configuring CTS and how to use Terraform providers and modules for tasks.](/docs/nia/installation/configure)
### Partner Terraform Modules
The modules listed below are available to use and are compatible with Consul-Terraform-Sync.
The modules listed below are available to use and are compatible with CTS.
#### A10 Networks

11
website/content/docs/nia/installation/run.mdx
View File

@ -2,7 +2,7 @@
layout: docs
page_title: Run Consul-Terraform-Sync
description: >-
Consul-Terraform-Sync requires a Terraform Provider, a Terraform Module and a running Consul Cluster outside of the consul-terraform-sync daemon.
Consul-Terraform-Sync requires a Terraform Provider, a Terraform Module and a running Consul Cluster outside of the `consul-terraform-sync` daemon.
---
# Run Consul-Terraform-Sync
@ -15,7 +15,7 @@ description: >-
2. Create the config.hcl file, all the options are available [here](/docs/nia/configuration).
3. Run consul-terraform-sync.
3. Run Consul-Terraform-Sync (CTS).
```shell-session
$ consul-terraform-sync -config-file <config.hcl>
@ -29,9 +29,6 @@ description: >-
## Other Run modes
Consul-Terraform-Sync allows you to inspect your configuration before applying
any change and to run in once mode, meaning that you can verify the changes are
correctly applied in a test run before running it in unsupervised daemon mode.
CTS allows you to inspect your configuration before applying any change and to run in once mode, meaning that you can verify the changes are correctly applied in a test run before running it in unsupervised daemon mode.
To learn more on these options check the
[Consul-Terraform-Sync Run Modes and Status Inspection](https://learn.hashicorp.com/tutorials/consul/consul-terraform-sync-run-and-inspect?utm_source=WEBSITE&utm_medium=WEB_IO&utm_offer=ARTICLE_PAGE&utm_content=DOCS) tutorial.
To learn more on these options check the [Consul-Terraform-Sync Run Modes and Status Inspection](https://learn.hashicorp.com/tutorials/consul/consul-terraform-sync-run-and-inspect?utm_source=WEBSITE&utm_medium=WEB_IO&utm_offer=ARTICLE_PAGE&utm_content=DOCS) tutorial.

0
website/content/docs/nia/lorna.txt Normal file
View File

16
website/content/docs/nia/network-drivers/index.mdx
View File

@ -7,26 +7,26 @@ description: >-
# Network Drivers
Consul-Terraform-Sync uses network drivers to execute and update network infrastructure. Drivers transform Consul service-level information into downstream changes by processing and abstracting API and resource details tied to specific network infrastructure.
Consul-Terraform-Sync (CTS) uses network drivers to execute and update network infrastructure. Drivers transform Consul service-level information into downstream changes by processing and abstracting API and resource details tied to specific network infrastructure.
Consul-Terraform-Sync is a HashiCorp solution to Network Infrastructure Automation. It bridges Consul's networking features and Terraform infrastructure management capabilities. The solution seamlessly embeds Terraform as network drivers to manage automation of Terraform modules. This expands the Consul ecosystem and taps into the rich features and community of Terraform and Terraform providers.
CTS is a HashiCorp solution to Network Infrastructure Automation. It bridges Consul's networking features and Terraform infrastructure management capabilities. The solution seamlessly embeds Terraform as network drivers to manage automation of Terraform modules. This expands the Consul ecosystem and taps into the rich features and community of Terraform and Terraform providers.
The following table highlights some of the additional features Terraform and Terraform Cloud offer when used as a network driver for Consul-Terraform-Sync. Visit the [Terraform product page](https://www.hashicorp.com/products/terraform) or [contact our sales team](https://www.hashicorp.com/contact-sales) for a comprehensive list of features.
The following table highlights some of the additional features Terraform and Terraform Cloud offer when used as a network driver for CTS. Visit the [Terraform product page](https://www.hashicorp.com/products/terraform) or [contact our sales team](https://www.hashicorp.com/contact-sales) for a comprehensive list of features.
| Network Driver | Description | Features |
| -------------- | ----------- | -------- |
| [Terraform driver](/docs/nia/network-drivers/terraform) | Consul-Terraform-Sync automates a local installation of the [Terraform CLI](https://www.terraform.io/) | - Local Terraform execution <br/> - Local workspace directories <br/> - [Backend options](/docs/nia/configuration#backend) available for state storage <br/> |
| [Terraform Cloud driver](/docs/nia/network-drivers/terraform-cloud) | Consul-Terraform-Sync Enterprise automates remote workspaces on [Terraform Cloud](https://www.terraform.io/docs/cloud/index.html) | - [Remote Terraform execution](https://www.terraform.io/docs/cloud/run/index.html) <br/> - Concurrent runs <br/> - [Secured variables](https://www.terraform.io/docs/cloud/workspaces/variables.html) <br/> - [State versions](https://www.terraform.io/docs/cloud/workspaces/state.html) <br/> - [Sentinel](https://www.terraform.io/docs/cloud/sentinel/index.html) to enforce governance policies as code <br/> - Audit [logs](https://www.terraform.io/docs/enterprise/admin/logging.html) and [trails](https://www.terraform.io/docs/cloud/api/audit-trails.html) <br/> - Run [history](https://www.terraform.io/docs/cloud/run/manage.html), [triggers](https://www.terraform.io/docs/cloud/workspaces/run-triggers.html), and [notifications](https://www.terraform.io/docs/cloud/workspaces/notifications.html) <br/> - [Terraform Cloud Agents](https://www.terraform.io/docs/cloud/agents/index.html) |
| [Terraform driver](/docs/nia/network-drivers/terraform) | CTS automates a local installation of the [Terraform CLI](https://www.terraform.io/) | - Local Terraform execution <br/> - Local workspace directories <br/> - [Backend options](/docs/nia/configuration#backend) available for state storage <br/> |
| [Terraform Cloud driver](/docs/nia/network-drivers/terraform-cloud) | CTS Enterprise automates remote workspaces on [Terraform Cloud](https://www.terraform.io/docs/cloud/index.html) | - [Remote Terraform execution](https://www.terraform.io/docs/cloud/run/index.html) <br/> - Concurrent runs <br/> - [Secured variables](https://www.terraform.io/docs/cloud/workspaces/variables.html) <br/> - [State versions](https://www.terraform.io/docs/cloud/workspaces/state.html) <br/> - [Sentinel](https://www.terraform.io/docs/cloud/sentinel/index.html) to enforce governance policies as code <br/> - Audit [logs](https://www.terraform.io/docs/enterprise/admin/logging.html) and [trails](https://www.terraform.io/docs/cloud/api/audit-trails.html) <br/> - Run [history](https://www.terraform.io/docs/cloud/run/manage.html), [triggers](https://www.terraform.io/docs/cloud/workspaces/run-triggers.html), and [notifications](https://www.terraform.io/docs/cloud/workspaces/notifications.html) <br/> - [Terraform Cloud Agents](https://www.terraform.io/docs/cloud/agents/index.html) |
## Understanding Terraform Automation
Consul-Terraform-Sync automates Terraform execution using a templated configuration to carry out infrastructure changes. The auto-generated configuration leverages input variables sourced from Consul and builds on top of reusable Terraform modules published and maintained by HashiCorp partners and the community. Consul-Terraform-Sync can also run your custom built modules that suit your team's specific network automation needs.
CTS automates Terraform execution using a templated configuration to carry out infrastructure changes. The auto-generated configuration leverages input variables sourced from Consul and builds on top of reusable Terraform modules published and maintained by HashiCorp partners and the community. CTS can also run your custom built modules that suit your team's specific network automation needs.
The network driver for Consul-Terraform-Sync determines how the Terraform automation operates. Visit the driver pages to read more about the [Terraform driver](/docs/nia/network-drivers/terraform) and the [Terraform Cloud driver](/docs/nia/network-drivers/terraform-cloud).
The network driver for CTS determines how the Terraform automation operates. Visit the driver pages to read more about the [Terraform driver](/docs/nia/network-drivers/terraform) and the [Terraform Cloud driver](/docs/nia/network-drivers/terraform-cloud).
### Upgrading Terraform
Upgrading the Terraform version used by Consul-Terraform-Sync may introduce breaking changes that can impact the Terraform modules. Refer to the Terraform [upgrade guides](https://www.terraform.io/upgrade-guides/index.html) for details before upgrading.
Upgrading the Terraform version used by CTS may introduce breaking changes that can impact the Terraform modules. Refer to the Terraform [upgrade guides](https://www.terraform.io/upgrade-guides/index.html) for details before upgrading.
The following versions were identified as containing changes that may impact Terraform modules.

47
website/content/docs/nia/network-drivers/terraform-cloud.mdx
View File

@ -12,16 +12,15 @@ description: >-
which is available with <strong>Consul Enterprise</strong>.
</EnterpriseAlert>
Consul-Terraform-Sync is more powerful when you integrate it with [Terraform Cloud](https://www.terraform.io/cloud). Integrating with Terraform Cloud provides features, such as enhanced workspaces and insight into Terraform operations as Consul-Terraform-Sync dynamically updates your network infrastructure. Consul-Terraform-Sync is compatible with both the [self-hosted](https://www.hashicorp.com/products/terraform/editions/enterprise) and [managed service](https://www.hashicorp.com/products/terraform/editions/cloud) versions of Terraform Cloud. It also supports all [tiers](https://www.hashicorp.com/products/terraform/pricing) of the Terraform Cloud managed service.
Consul-Terraform-Sync (CTS) is more powerful when you integrate it with [Terraform Cloud](https://www.terraform.io/cloud). Integrating with Terraform Cloud provides features, such as enhanced workspaces and insight into Terraform operations as CTS dynamically updates your network infrastructure. CTS is compatible with both the [self-hosted](https://www.hashicorp.com/products/terraform/editions/enterprise) and [managed service](https://www.hashicorp.com/products/terraform/editions/cloud) versions of Terraform Cloud. It also supports all [tiers](https://www.hashicorp.com/products/terraform/pricing) of the Terraform Cloud managed service.
This page describes how the Terraform Cloud driver operates within Consul-Terraform-Sync (CTS).
This page describes how the Terraform Cloud driver operates within CTS.
## Terraform Workspace Automation
Consul-Terraform-Sync manages Terraform runs following the [API-driven run workflow](https://www.terraform.io/docs/cloud/run/api.html) for workspaces in Terraform Cloud.
CTS manages Terraform runs following the [API-driven run workflow](https://www.terraform.io/docs/cloud/run/api.html) for workspaces in Terraform Cloud.
On startup, Consul-Terraform-Sync:
On startup, CTS:
1. Creates or discovers Terraform Cloud workspaces corresponding to the configured tasks.
2. Prepares the local environment and generates Terraform configuration files that make up the root module for each task.
3. Packages the generated files and uploads them as a configuration version for the task's workspace on Terraform Cloud.
@ -32,17 +31,17 @@ Once all workspaces are set up, CTS monitors the Consul catalog for service chan
## Remote Workspaces
Consul-Terraform-Sync will discover or create a new workspaces based on your configured tasks. The task configuration options for `name`, `description`, and `terraform_version` are used to set the workspace name, description, and Terraform version.
CTS will discover or create a new workspaces based on your configured tasks. The task configuration options for `name`, `description`, and `terraform_version` are used to set the workspace name, description, and Terraform version.
[![CTS Workspace Overview](/img/nia/cts-tfc-workspace.png)](/img/nia/cts-tfc-workspace.png)
Workspace automation requirements for Consul-Terraform-Sync are in place to avoid overriding other workspaces unintentionally.
Workspace automation requirements for CTS are in place to avoid overriding other workspaces unintentionally.
* Must be set to remote execution mode
* Cannot be connected to a VCS
* Cannot have an existing configuration version uploaded by another application
* Must satisfy workspace [tag requirements](/docs/nia/configuration#tags_allowlist) and [tag restrictions](/docs/nia/configuration#tags_denylist) set by the CTS operator
Workspaces created by Consul-Terraform-Sync will be configured with the following settings:
Workspaces created by CTS will be configured with the following settings:
| Setting | Value |
| ------- | ----- |
@ -53,11 +52,11 @@ Workspaces created by Consul-Terraform-Sync will be configured with the followin
| Terraform Version | [`task.terraform_version`](/docs/nia/configuration#terraform_version) or the latest [Terraform version compatible with CTS](/docs/nia/compatibility#terraform) available for the organization. |
| Tags | `source:cts` and [additional tags](/docs/nia/configuration#tags) set by the CTS operator |
Other workspace settings can be pre-configured or updated, such as setting the workspace to [manual apply](#manual-apply) or adding a [run notification](https://www.terraform.io/docs/cloud/workspaces/notifications.html) to send messages to a Slack channel when Consul-Terraform-Sync updates your network infrastructure.
Other workspace settings can be pre-configured or updated, such as setting the workspace to [manual apply](#manual-apply) or adding a [run notification](https://www.terraform.io/docs/cloud/workspaces/notifications.html) to send messages to a Slack channel when CTS updates your network infrastructure.
### Manual Apply
Consul-Terraform-Sync (CTS) can automate remote workspaces with either auto apply or manual apply configured. Having CTS manage workspaces with manual apply is useful to add an approval stage to CTS automation. Operators can manually inspect and approve or discard runs that CTS had queued based on the task run condition.
CTS can automate remote workspaces with either auto apply or manual apply configured. Having CTS manage workspaces with manual apply is useful to add an approval stage to CTS automation. Operators can manually inspect and approve or discard runs that CTS had queued based on the task run condition.
When CTS detects new changes for a workspace that already has a run pending on approval, CTS will discard the stale run and queue a new run with the latest values. The new run will go through plan and then again wait on an operator to approve it. Only once the run is approved will the infrastructure be updated with the latest Consul changes.
@ -69,7 +68,7 @@ There are two approaches to setup manual apply for a workspace managed by CTS ba
1. Ensure that the apply method for the workspace is set to manual apply.
1. Configure the task for the workspace and run CTS.
-> **Tip**: Setup [run notifications](https://www.terraform.io/docs/cloud/workspaces/notifications.html#creating-a-notification-configuration) for workspaces with manual apply to not miss automated runs by Consul-Terraform-Sync. Look into setting the [buffer period](/docs/nia/configuration#buffer_period-1) or a [schedule condition](/docs/nia/configuration#schedule-condition) to group changes together and reduce runs requiring approval.
-> **Tip**: Setup [run notifications](https://www.terraform.io/docs/cloud/workspaces/notifications.html#creating-a-notification-configuration) for workspaces with manual apply to not miss automated runs by CTS. Look into setting the [buffer period](/docs/nia/configuration#buffer_period-1) or a [schedule condition](/docs/nia/configuration#schedule-condition) to group changes together and reduce runs requiring approval.
## Configuration Version
@ -86,54 +85,54 @@ sync-tasks/
- `main.tf` - The main file contains the terraform block, provider blocks, and a module block calling the module configured for the task.
- `terraform` block - The corresponding provider source and versions for the task from the configuration files are placed into this block for the root module.
- `provider` blocks - The provider blocks generated in the root module resemble the `terraform_provider` blocks from the configuration for Consul-Terraform-Sync. They have identical arguments present and are set from the intermediate variable created per provider.
- `provider` blocks - The provider blocks generated in the root module resemble the `terraform_provider` blocks from the configuration for CTS. They have identical arguments present and are set from the intermediate variable created per provider.
- `module` block - The module block is where the task's module is called as a [child module](https://www.terraform.io/docs/configuration/modules.html#calling-a-child-module). The child module contains the core logic for automation. Required and optional input variables are passed as arguments to the module.
- `variables.tf` - This file contains three types of variable declarations:
- `services` input variable (required) determines module compatibility with Consul-Terraform Sync (read more on [compatible Terraform modules](/docs/nia/terraform-modules) for more details).
- Any additional [optional input variables](/docs/nia/terraform-modules#optional-input-variables) provided by Consul-Terraform-Sync that the module may include.
- Various intermediate variables used to configure providers. Intermediate provider variables are interpolated from the provider blocks and arguments configured in the Consul-Terraform-Sync configuration.
- Any additional [optional input variables](/docs/nia/terraform-modules#optional-input-variables) provided by CTS that the module may use.
- Various intermediate variables used to configure providers. Intermediate provider variables are interpolated from the provider blocks and arguments configured in the CTS configuration.
- `variables.module.tf` - This file is created if there are [variables configured for the task](/docs/nia/configuration#variable_files) and contains the interpolated variable declarations that match the variables from configuration. These are then used to proxy the configured variables to the module through explicit assignment in the module block.
## Variables
Consul-Terraform-Sync uses Terraform input variables to reflect the latest Consul service information. They are used as parameters for your Terraform module. Input variables are dynamic and are updated by the driver throughout the runtime of Consul-Terraform-Sync.
CTS uses Terraform input variables to reflect the latest Consul service information. They are used as parameters for your Terraform module. Input variables are dynamic and are updated by the driver throughout the runtime of CTS.
You can view the latest service information in the Terraform UI by navigating to that workspace and clicking the "Variables" tab in the workspace navigation.
[![CTS Workspace Variables](/img/nia/cts-tfc-workspace-variables.png)](/img/nia/cts-tfc-workspace-variables.png)
~> **Caution:** Dynamic variables maintained by Consul-Terraform-Sync are formatted for automation. Unexpected manual changes to these variables may result in automation errors.
~> **Caution:** Dynamic variables maintained by CTS are formatted for automation. Unexpected manual changes to these variables may result in automation errors.
## Setting Up Terraform Cloud Driver
### Deployment
Because a Consul-Terraform-Sync instance can only be configured with one driver, an instance can only be associated with either a Terraform driver or a Terraform Cloud driver. If there is a need to run both types of drivers, users will need to deploy a separate Consul-Terraform-Sync instance for each type of driver. Relatedly, if there is a need to run Consul-Terraform-Sync across multiple Terraform Cloud organizations, users will need to deploy a separate instance for each organization.
Because a CTS instance can only be configured with one driver, an instance can only be associated with either a Terraform driver or a Terraform Cloud driver. If there is a need to run both types of drivers, users will need to deploy a separate CTS instance for each type of driver. Relatedly, if there is a need to run CTS across multiple Terraform Cloud organizations, users will need to deploy a separate instance for each organization.
### Required Setup
This section captures requirements for setting up Consul-Terraform-Sync to integrate with your [Terraform Cloud](https://www.hashicorp.com/cloud) solution.
This section captures requirements for setting up CTS to integrate with your [Terraform Cloud](https://www.hashicorp.com/cloud) solution.
1. Hostname of your Terraform Cloud, self-hosted distribution
1. Name of your organization
1. [Team API token](https://www.terraform.io/docs/cloud/users-teams-organizations/api-tokens.html#team-api-tokens) used for authentication with Terraform Cloud
Prior to running Consul-Terraform-Sync with a Terraform Cloud driver, you will need an account and organization set up, as well as a dedicated token. We recommend using a team token that is restricted to [Manage Workspaces](https://www.terraform.io/docs/cloud/users-teams-organizations/teams.html#managing-workspace-access)-level permissions. Below are the steps for the recommended setup.
Prior to running CTS with a Terraform Cloud driver, you will need an account and organization set up, as well as a dedicated token. We recommend using a team token that is restricted to [Manage Workspaces](https://www.terraform.io/docs/cloud/users-teams-organizations/teams.html#managing-workspace-access)-level permissions. Below are the steps for the recommended setup.
The first step is to create an account with your Terraform Cloud service. After creating an account, create a new [organization](https://www.terraform.io/docs/cloud/users-teams-organizations/organizations.html#creating-organizations) or select an existing organization. The address of your Terraform Cloud service will be used to configure the [`hostname`](/docs/nia/configuration#hostname), and the organization name will be used to configure the [`organization`](/docs/nia/configuration#organization) on the Terraform Cloud driver.
Once you have an account and organization, the next step is to [create a team](https://www.terraform.io/docs/cloud/users-teams-organizations/teams.html). We recommend using a dedicated team and team token to run and authenticate Consul-Terraform-Sync. Using a team token has the benefits of restricting organization permissions as well as associating Consul-Terraform-Sync automated actions with the team rather than an individual.
Once you have an account and organization, the next step is to [create a team](https://www.terraform.io/docs/cloud/users-teams-organizations/teams.html). We recommend using a dedicated team and team token to run and authenticate CTS. Using a team token has the benefits of restricting organization permissions as well as associating CTS automated actions with the team rather than an individual.
After creating a dedicated team, update the team's permissions with "Manage Workspaces" organization access-level. Consul-Terraform-Sync's main work revolves around creating and managing workspaces. Therefore restricting the dedicated team's permission to Manage Workspaces level is sufficient and reduces security risk.
After creating a dedicated team, update the team's permissions with "Manage Workspaces" organization access-level. CTS's main work revolves around creating and managing workspaces. Therefore restricting the dedicated team's permission to Manage Workspaces level is sufficient and reduces security risk.
[![CTS Terraform Team Setup](/img/nia/cts-tfc-team-setup.png)](/img/nia/cts-tfc-team-setup.png)
After setting the team's permissions, the final setup step is to [generate the associated team token](https://www.terraform.io/docs/cloud/users-teams-organizations/api-tokens.html#team-api-tokens), which can be done on the same team management page. This token will be used by Consul-Terraform-Sync for API authentication and will be used to configure the [`token`](/docs/nia/configuration#token-1) on the Terraform Cloud driver.
After setting the team's permissions, the final setup step is to [generate the associated team token](https://www.terraform.io/docs/cloud/users-teams-organizations/api-tokens.html#team-api-tokens), which can be done on the same team management page. This token will be used by CTS for API authentication and will be used to configure the [`token`](/docs/nia/configuration#token-1) on the Terraform Cloud driver.
### Recommendations
We recommend configuring workspaces managed by Consul-Terraform-Sync with [run notifications](https://www.terraform.io/docs/cloud/workspaces/notifications.html) through the Terraform web application. Run notifications notify external systems about the progress of runs and could help notify users of Consul-Terraform-Sync events, particularly errored runs.
We recommend configuring workspaces managed by CTS with [run notifications](https://www.terraform.io/docs/cloud/workspaces/notifications.html) through the Terraform web application. Run notifications notify external systems about the progress of runs and could help notify users of CTS events, particularly errored runs.
[![CTS Terraform Cloud Run Notifications](/img/nia/cts-tfc-run-notifications.png)](/img/nia/cts-tfc-run-notifications.png)
In order to configure a run notification, users can [manually create a notification configuration](https://www.terraform.io/docs/cloud/workspaces/notifications.html#creating-a-notification-configuration) for workspaces automated by Consul-Terraform-Sync. A workspace may already exist for a task if the workspace name is identical to the configured task's [`name`](/docs/nia/configuration#name-2). This may occur if Consul-Terraform-Sync has already already run and created the workspace for the task. This may also occur if the workspace is manually created for the task prior to Consul-Terraform-Sync running.
In order to configure a run notification, users can [manually create a notification configuration](https://www.terraform.io/docs/cloud/workspaces/notifications.html#creating-a-notification-configuration) for workspaces automated by CTS. A workspace may already exist for a task if the workspace name is identical to the configured task's [`name`](/docs/nia/configuration#name-2). This may occur if CTS has already already run and created the workspace for the task. This may also occur if the workspace is manually created for the task prior to CTS running.

26
website/content/docs/nia/network-drivers/terraform.mdx
View File

@ -7,28 +7,28 @@ description: >-
# Terraform Driver
Consul-Terraform-Sync extends the Consul ecosystem to include Terraform as an officially supported tooling project. With the Terraform driver, Consul-Terraform-Sync installs the [Terraform CLI](https://www.terraform.io/downloads.html) locally and runs Terraform commands based on monitored Consul changes. This page details how the Terraform driver operates using local workspaces and templated files.
Consul-Terraform-Sync (CTS) extends the Consul ecosystem to include Terraform as an officially supported tooling project. With the Terraform driver, CTS installs the [Terraform CLI](https://www.terraform.io/downloads.html) locally and runs Terraform commands based on monitored Consul changes. This page details how the Terraform driver operates using local workspaces and templated files.
## Terraform CLI Automation
On startup, Consul-Terraform-Sync:
On startup, CTS:
1. Downloads and installs Terraform
2. Prepares local workspace directories. Terraform configuration and execution for each task is organized as separate [Terraform workspaces](https://www.terraform.io/docs/state/workspaces.html). The state files for tasks are independent of each other.
3. Generates Terraform configuration files that make up the root module for each task.
Once all workspaces are set up, Consul-Terraform-Sync monitors the Consul catalog for service changes. When relevant changes are detected, the Terraform driver dynamically updates input variables for that task using a template to render them to a file named [`terraform.tfvars`](/docs/nia/network-drivers#terraform-tfvars). This file is passed as a parameter to the Terraform CLI when executing `terraform plan` and `terraform apply` to update your network infrastructure with the latest Consul service details.
Once all workspaces are set up, CTS monitors the Consul catalog for service changes. When relevant changes are detected, the Terraform driver dynamically updates input variables for that task using a template to render them to a file named [`terraform.tfvars`](/docs/nia/network-drivers#terraform-tfvars). This file is passed as a parameter to the Terraform CLI when executing `terraform plan` and `terraform apply` to update your network infrastructure with the latest Consul service details.
### Local Workspaces
Within the Consul-Terraform-Sync configuration for a task, practitioners can select the desired module to run for the task as well as set the condition to execute the task. Each task executed by the Terraform driver corresponds to an automated root module that calls the selected module in an isolated Terraform environment. Consul-Terraform-Sync will manage concurrent execution of these tasks.
Within the CTS configuration for a task, practitioners can select the desired module to run for the task as well as set the condition to execute the task. Each task executed by the Terraform driver corresponds to an automated root module that calls the selected module in an isolated Terraform environment. CTS will manage concurrent execution of these tasks.
Autogenerated root modules for tasks are maintained in local subdirectories of the Consul-Terraform-Sync working directory. Each subdirectory represents the local workspace for a task. By default, the working directory `sync-tasks` is created in the current directory. To configure where Terraform configuration files are stored, set [`working_dir`](/docs/nia/configuration#working_dir) to the desired path or configure the [`task.working_dir`](/docs/nia/configuration#working_dir-1) individually.
Autogenerated root modules for tasks are maintained in local subdirectories of the CTS working directory. Each subdirectory represents the local workspace for a task. By default, the working directory `sync-tasks` is created in the current directory. To configure where Terraform configuration files are stored, set [`working_dir`](/docs/nia/configuration#working_dir) to the desired path or configure the [`task.working_dir`](/docs/nia/configuration#working_dir-1) individually.
~> **Note:** Although Terraform state files for task workspaces are independent, this does not guarantee the infrastructure changes from concurrent task executions are independent. Ensure that modules across all tasks are not modifying the same resource objects or have overlapping changes that may result in race conditions during automation.
### Root Module
The root module proxies Consul information, configuration, and other variables to the Terraform module for the task. The content of the files that make up the root module are sourced from Consul-Terraform-Sync configuration, information for task's module to use as the automation playbook, and information from Consul such as service information.
The root module proxies Consul information, configuration, and other variables to the Terraform module for the task. The content of the files that make up the root module are sourced from CTS configuration, information for task's module to use as the automation playbook, and information from Consul such as service information.
A working directory with one task named "cts-example" would have the folder structure below when running with the Terraform driver.
@ -43,19 +43,19 @@ sync-tasks/
└── terraform.tfvars.tmpl
```
The following files of the root module are generated for each task. An [example of a root module created by Consul-Terraform-Sync](https://github.com/hashicorp/consul-terraform-sync/tree/master/examples) can be found in the project repository.
The following files of the root module are generated for each task. An [example of a root module created by CTS](https://github.com/hashicorp/consul-terraform-sync/tree/master/examples) can be found in the project repository.
- `main.tf` - The main file contains the terraform block, provider blocks, and a module block calling the module configured for the task.
- `terraform` block - The corresponding provider source and versions for the task from the configuration files are placed into this block for the root module. The Terraform backend from the configuration is also templated here.
- `provider` blocks - The provider blocks generated in the root module resemble the `terraform_provider` blocks from the configuration for Consul-Terraform-Sync. They have identical arguments present and are set from the intermediate variable created per provider.
- `provider` blocks - The provider blocks generated in the root module resemble the `terraform_provider` blocks from the configuration for CTS. They have identical arguments present and are set from the intermediate variable created per provider.
- `module` block - The module block is where the task's module is called as a [child module](https://www.terraform.io/docs/configuration/modules.html#calling-a-child-module). The child module contains the core logic for automation. Required and optional input variables are passed as arguments to the module.
- `variables.tf` - This file contains three types of variable declarations.
- `services` input variable (required) determines module compatibility with Consul-Terraform Sync (read more on [compatible Terraform modules](/docs/nia/terraform-modules) for more details).
- Any additional [optional input variables](/docs/nia/terraform-modules#optional-input-variables) provided by Consul-Terraform-Sync that the module may include.
- Various intermediate variables used to configure providers. Intermediate provider variables are interpolated from the provider blocks and arguments configured in the Consul-Terraform-Sync configuration.
- Any additional [optional input variables](/docs/nia/terraform-modules#optional-input-variables) provided by CTS that the module may use.
- Various intermediate variables used to configure providers. Intermediate provider variables are interpolated from the provider blocks and arguments configured in the CTS configuration.
- `variables.module.tf` - This file is created if there are [variables configured for the task](/docs/nia/configuration#variable_files) and contains the interpolated variable declarations that match the variables from configuration. These are then used to proxy the configured variables to the module through explicit assignment in the module block.
- `providers.tfvars` - This file is created if there are [providers configured for the task](/docs/nia/configuration#providers) and defined [`terraform_provider` blocks](/docs/nia/configuration#terraform-provider). This file may contain sensitive information. To omit sensitive information from this file, you can [securely configure Terraform providers for Consul-Terraform-Sync](/docs/nia/configuration#securely-configure-terraform-providers) using environment variables or templating.
- `terraform.tfvars` - The variable definitions file is where the services input variable and any optional Consul-Terraform-Sync input variables are assigned values from Consul. It is periodically updated, typically when the task condition is met, to reflect the current state of Consul.
- `terraform.tfvars.tmpl` - The template file is used by Consul-Terraform-Sync to template information from Consul by using the HashiCorp configuration and templating library ([hashicorp/hcat](https://github.com/hashicorp/hcat)).
- `providers.tfvars` - This file is created if there are [providers configured for the task](/docs/nia/configuration#providers) and defined [`terraform_provider` blocks](/docs/nia/configuration#terraform-provider). This file may contain sensitive information. To omit sensitive information from this file, you can [securely configure Terraform providers for CTS](/docs/nia/configuration#securely-configure-terraform-providers) using environment variables or templating.
- `terraform.tfvars` - The variable definitions file is where the services input variable and any optional CTS input variables are assigned values from Consul. It is periodically updated, typically when the task condition is met, to reflect the current state of Consul.
- `terraform.tfvars.tmpl` - The template file is used by CTS to template information from Consul by using the HashiCorp configuration and templating library ([hashicorp/hcat](https://github.com/hashicorp/hcat)).
~> **Note:** Generated template and Terraform configuration files are crucial for the automation of tasks. Any manual changes to the files may not be preserved and could be overwritten by a subsequent update. Unexpected manual changes to the format of the files may cause automation to error.

360
website/content/docs/nia/release-notes/0-5-0.mdx Normal file
View File

@ -0,0 +1,360 @@
---
layout: docs
page_title: CTS v0.5.0
description: >-
Consul-Terraform-Sync release notes for v0.5.0
---
-> The release notes for v0.5.0 focus on a subset of configuration deprecations
# Configuration Deprecations in v0.5
We introduced a number of configuration deprecations in Consul-Terraform-Sync (CTS) v0.5. Many of these deprecations are part of an effort to streamline CTS task configuration and reduce the inconsistencies. As CTS support for more use-cases increased, inconsistencies arose and required a lot of documentation to explain.
This documentation outlines the deprecations and examples on how to upgrade your configuration for v0.5. The deprecated configurations will not be removed until v0.8 and later. We plan to also release a tool to automate upgrading your configurations to make the upgrade quicker and smoother.
## Overview of Deprecations
The deprecations address two major inconsistencies. First is the use of the terminology "source" in various configuration blocks and fields. The second is the exception to configuring a task condition or a task module input by using the `services` field and its related `service` block.
### Address "source" terminology
The word "source" is used in the name of a number of task configurations. "Source" in the configuration name commonly refers to the Terraform module though it is not immediately clear and requires additional documentation to explain.
We deprecated all configuration with the term "source" and replaced it with "module" in the task configuration:
| Deprecated Config | Replacement Config | Description |
| --- | --- | --- |
| `source` field | `module` field | The path to the Terraform module |
| `source_input` block | `module_input` block | The configuration block to define the values for additional Terraform input variables for the Terraform module |
| `source_includes_var` field | `use_as_module_input` field | The field on the `condition` block to indicate whether the values of the condition object should also be used as module input |
Removal date:
- `source`: future major release after v0.8
- `source_input`: v0.8
- `source_includes_var`: v0.8
The documentation below will refer to the deprecated configurations by their new name.
### Address `services` field and `service` block
The `services` field and its associated `service` block are frequently an exception to task configuration use-cases and add complexity and restrictions.
The `services` field is the only task condition that is not configured through the `condition` block. It is also the only additional module input that is not configured through a `module_input` block. This leads to a lot of inconsistent relationships with other `condition` and `module_input` blocks.
We deprecated the `services` field and its associated `service` filter to remove these exceptions and complicated relationship with other `condition` and `module_input` blocks:
| Deprecated Config | Replacement Config | Description |
| --- | --- | --- |
| `services` field | `condition "services"` or `module_input "services"` block | The services to provide module input and occasionally act as the task condition |
| `service` block | fields moved into the `condition "services"` and `module_input "services"` block | The configuration block to provide additional filtering on the services in the `services` field |
The `services` field and its associated `service` block will be removed in a future major release after v0.8.
## Further Details on Deprecations
### Deprecate `source` field
The `source` field in the task configuration is deprecated and can be directly renamed to `module`.
Example:
<CodeTabs heading="Deprecate source field" tabs={["Deprecated", "New"]}>
<CodeBlockConfig highlight="4">
```hcl
task {
name = "services_task"
services = ["api", "web"]
source = "path/to/module"
}
```
</CodeBlockConfig>
<CodeBlockConfig highlight="4">
```hcl
task {
name = "services_task"
services = ["api", "web"]
module = "path/to/module"
}
```
</CodeBlockConfig>
</CodeTabs>
### Deprecate `source_input` block
The `source_input` block in the task configuration is deprecated and can be directly renamed to `module_input`.
Example:
<CodeTabs heading="Deprecate source_input block" tabs={["Deprecated", "New"]}>
<CodeBlockConfig highlight="7">
```hcl
task {
name = "scheduled_task"
module = "path/to/module"
condition "schedule" {
cron = "* * * * Mon"
}
source_input "consul-kv" {
path = "my_key"
}
}
```
</CodeBlockConfig>
<CodeBlockConfig highlight="7">
```hcl
task {
name = "scheduled_task"
module = "path/to/module"
condition "schedule" {
cron = "* * * * Mon"
}
module_input "consul-kv" {
path = "my_key"
}
}
```
</CodeBlockConfig>
</CodeTabs>
### Deprecate `source_includes_var` field
The `condition` block's `source_includes_var` field is deprecated for all types of conditions and can be directly renamed to `use_as_module_input`.
Example:
<CodeTabs heading="Deprecate source_includes_var field" tabs={["Deprecated", "New"]}>
<CodeBlockConfig highlight="6">
```hcl
task {
name = "catalog_services_task"
module = "path/to/module"
condition "catalog-services" {
regexp = "api"
source_includes_var = true
}
}
```
</CodeBlockConfig>
<CodeBlockConfig highlight="6">
```hcl
task {
name = "catalog_services_task"
module = "path/to/module"
condition "catalog-services" {
regexp = "api"
use_as_module_input = true
}
}
```
</CodeBlockConfig>
</CodeTabs>
### Deprecate `services` field
The `services` field can play different roles in a task depending on other task configurations:
- When no `condition` block is configured, then the `services` field is the task's condition and module input
- When there is a `condition` block configured, then the `services field is only module input for the task
The `services` field can be replaced depending on the role it has for the task.
**Deprecate `services` field as a condition**
When the `services` field behaves as a condition, it can be replaced with a `condition "services"` block with the `names` field set to the list of the services.
Example:
<CodeTabs heading="Deprecate services field as a condition" tabs={["Deprecated", "New"]}>
<CodeBlockConfig highlight="4">
```hcl
task {
name = "services_condition_task"
module = "path/to/module"
services = ["api", "web"]
}
```
</CodeBlockConfig>
<CodeBlockConfig highlight="4,5,6">
```hcl
task {
name = "services_condition_task"
module = "path/to/module"
condition "services" {
names = ["api", "web"]
}
}
```
</CodeBlockConfig>
</CodeTabs>
**Deprecate `services` field as module input only**
When the `services` field only provides module input, it can be replaced with a `module_input "services"` block with the `names` field set to the list of the services.
Example:
<CodeTabs heading="Deprecate services field as a module input" tabs={["Deprecated", "New"]}>
<CodeBlockConfig highlight="8">
```hcl
task {
name = "services_module_input_task"
module = "path/to/module"
condition "consul-kv" {
path = "my_key"
source_includes_var = true
}
services = ["api", "web"]
}
```
</CodeBlockConfig>
<CodeBlockConfig highlight="8,9,10">
```hcl
task {
name = "services_module_input_task"
module = "path/to/module"
condition "consul-kv" {
path = "my_key"
use_as_module_input = true
}
module_input "services" {
names = ["api", "web"]
}
}
```
</CodeBlockConfig>
</CodeTabs>
-> Note: `use_as_module_input` was updated in v0.5 so that it will default to `true` when unconfigured. In the new configuration, the line to set `use_as_module_input = true` is no longer necessary
### Deprecate service block
The `service` block is a configuration block outside of the task block. The fields of the `service` block provides additional filtering to any task that has that service set in the `services` field.
Once the `services` field is upgraded to a `condition` or `module_input` block, as described in the section above, the deprecated `service` block's fields can be moved to the new `condition` or `module_input` block.
Example:
<CodeTabs heading="Deprecate service block" tabs={["Deprecated", "New"]}>
<CodeBlockConfig highlight="9,10,11,12,13">
```hcl
task {
name = "services_filter_task"
module = "path/to/module"
condition "services" {
names = ["api"]
}
}
service {
name = "api"
datacenter = "dc2"
namespace = "ns"
}
```
</CodeBlockConfig>
<CodeBlockConfig highlight="6,7">
```hcl
task {
name = "services_filter_task"
module = "path/to/module"
condition "services" {
names = ["api"]
datacenter = "dc2"
namespace = "ns"
}
}
```
</CodeBlockConfig>
</CodeTabs>
There is an edge-case that requires a more complicated deprecation upgrade. Some tasks may have multiple services in the `services` field and a `service` block configured for each of those services. When the fields are different across the `service` blocks, it is necessary to split the task and create new, separate tasks for each service.
Example:
<CodeTabs heading="Deprecate service block edge-case" tabs={["Deprecated", "New"]}>
<CodeBlockConfig highlight="2,4,7,8,9,10,11,12,13,14,15">
```hcl
task {
name = "services_task"
module = "path/to/module"
services = ["api", "web"]
}
service {
name = "api"
datacenter = "api_dc"
}
service {
name = "web"
datacenter = "web_dc"
}
```
</CodeBlockConfig>
<CodeBlockConfig highlight="2,4,5,6,7,11,13,14,15,16">
```hcl
task {
name = "api_services_task"
module = "path/to/module"
condition "services" {
names = ["api"]
datacenter = "api_dc"
}
}
task {
name = "web_services_task"
module = "path/to/module"
condition "services" {
names = ["web"]
datacenter = "web_dc"
}
}
```
</CodeBlockConfig>
</CodeTabs>

115
website/content/docs/nia/tasks.mdx
View File

@ -7,7 +7,7 @@ description: >-
# Tasks
A task is the translation of dynamic service information from the Consul Catalog into network infrastructure changes downstream. Consul-Terraform-Sync carries out automation for executing tasks using network drivers. For a Terraform driver, the scope of a task is a Terraform module.
A task is the translation of dynamic service information from the Consul Catalog into network infrastructure changes downstream. Consul-Terraform-Sync (CTS) carries out automation for executing tasks using network drivers. For a Terraform driver, the scope of a task is a Terraform module.
Below is an example task configuration:
@ -16,13 +16,15 @@ task {
name = "frontend-firewall-policies"
description = "Add firewall policy rules for frontend services"
providers = ["fake-firewall", "null"]
services = ["web", "image"]
source = "example/firewall-policy/module"
module = "example/firewall-policy/module"
version = "1.0.0"
condition "services" {
names = ["web", "image"]
}
}
```
In the example task above, the "fake-firewall" and "null" providers, listed in the `providers` field, are used. These providers themselves should be configured in their own separate [`terraform_provider` blocks](/docs/nia/configuration#terraform-provider). These providers are used in the Terraform module "example/firewall-policy/module", configured in the `source` field, to create, update, and destroy resources. This module may do something like use the providers to create and destroy firewall policy objects based on IP addresses. The IP addresses come from the "web" and "image" service instances configured in the `services` field. This service-level information is retrieved by Consul-Terraform-Sync which watches Consul catalog for changes.
In the example task above, the "fake-firewall" and "null" providers, listed in the `providers` field, are used. These providers themselves should be configured in their own separate [`terraform_provider` blocks](/docs/nia/configuration#terraform-provider). These providers are used in the Terraform module "example/firewall-policy/module", configured in the `module` field, to create, update, and destroy resources. This module may do something like use the providers to create and destroy firewall policy objects based on IP addresses. The IP addresses come from the "web" and "image" service instances configured in the `condition "services"` block. This service-level information is retrieved by CTS which watches Consul catalog for changes.
See [task configuration](/docs/nia/configuration#task) for more details on how to configure a task.
@ -32,19 +34,17 @@ A task can be either enabled or disabled using the [task cli](/docs/nia/cli/task
An enabled task can be configured to monitor and execute on different types of conditions, such as changes to services ([services condition](/docs/nia/tasks#services-condition)) or service registration and deregistration ([catalog-services condition](/docs/nia/tasks#catalog-services-condition)).
A task can also monitor, but not execute on, other variables that provide additional information to the task's module. For example, a task with a catalog-services condition may execute on registration changes, and monitor service instances for IP information.
A task can also monitor, but not execute on, other variables that provide additional information to the task's module. For example, a task with a catalog-services condition may execute on registration changes and additionally monitor service instances for IP information.
A source input can be specified that implicitly includes variables to be provided to the task’s module. For example, a task can specify a Consul KV source input. The specified KV keys or key paths would be monitored for changes. Any changes detected would be included as input information for the modules. The module determines the details of what values are monitored and what values can execute the task.
~> **The source input block is currently only supported when using a schedule condition.** Adding a source input block alongside any other type of condition will result in an error. To accomplish a similar behavior with other condition blocks, use the `source_includes_var` field.
Below are details on the types of execution conditions that Consul-Terraform-Sync supports.
All configured monitored information, regardless if it's used for execution or not, can be passed to the task's module as module input. Below are details on the types of execution conditions that CTS supports and their module inputs.
### Services Condition
The services condition is the default condition to execute a task. Tasks with the services condition monitor and execute on either changes to a list of configured services or changes to any services that match a given regex.
Tasks with the services condition monitor and execute on either changes to a list of configured services or changes to any services that match a given regex.
In order to configure a task with the services condition, the list of services to trigger the task must be configured in the task's [`services`](/docs/nia/configuration#services) or a regex must be configured in the task's `condition` block. Only one of these two options can be configured for a single task. See the [Services Condition](/docs/nia/configuration#services-condition) configuration section for more details.
There are two ways to configure a task with a services condition. Only one of the two options below can be configured for a single task:
1. Configure a task's [`services`](/docs/nia/configuration#services) field (deprecated) to specify the list of services to trigger the task
1. Configure a task's `condition` block with the [services condition](/docs/nia/configuration#services-condition) type to specify services to trigger the task.
The services condition operates by monitoring the [Health List Nodes For Service API](/api-docs/health#list-nodes-for-service) and executing the task on any change of information for services configured. These changes include one or more changes to service values, like IP address, added or removed service instance, or tags. A complete list of values that would cause a task to run are expanded below:
@ -65,43 +65,27 @@ The services condition operates by monitoring the [Health List Nodes For Service
| `node_tagged_addresses` | List of explicit LAN and WAN IP addresses for the agent |
| `node_meta` | List of user-defined metadata key/value pairs for the node |
The services condition is the default behavior if no `condition` block is configured for a task. The two tasks below have equivalent behavior, which is to execute when any of the listed services have changes to the previously stated attributes.
```hcl
task {
name = "services_condition_task_1"
description = "execute on changes to api, db, and web services"
providers = ["my-provider"]
source = "path/to/services-condition-module"
services = ["api", "db", "web"]
}
task {
name = "services_condition_task_2"
description = "execute on changes to api, db, and web services"
providers = ["my-provider"]
source = "path/to/services-condition-module"
services = ["api", "db", "web"]
condition "services" {}
}
```
Below is an example configuration for a task that will execute when a service with a name that matches the regular expression has a change. Note that because `regexp` is set, `task.services` is omitted.
Below is an example configuration for a task that will execute when a service with a name that matches the regular expression has a change.
```hcl
task {
name = "services_condition_task"
description = "execute on changes to services whose name starts with web"
providers = ["my-provider"]
source = "path/to/services-condition-module"
module = "path/to/services-condition-module"
condition "services" {
regexp = "^web.*"
regexp = "^web.*"
use_as_module_input = false
}
}
```
The services condition can provide input for the [`services` input variable](/docs/nia/terraform-modules#services-variable) that is required for each CTS module. This can be provided depending on how the services condition is configured:
- task's `services` field (deprecated): services object is automatically passed as module input
- task's `condition "services"` block: users can configure the `use_as_module_input` field to optionally use the condition's services object as module input
- Field was previously named `source_includes_var` (deprecated)
### Catalog-Services Condition
Tasks with a catalog-services condition monitor and execute on service registration changes for services that satisfy the condition configuration. 'Service registration changes' specifically refers to service registration and deregistration where service registration occurs on the first service instance registration, and service deregistration occurs on the last service instance registration. Tasks with a catalog-services condition may, depending on the module, additionally monitor but not execute on service instance information.
@ -113,28 +97,25 @@ Below is an example configuration for a task that will execute when a service wi
```hcl
task {
name = "catalog_service_condition_task"
source = "path/to/catalog-services-module"
module = "path/to/catalog-services-module"
providers = ["my-provider"]
services = ["web-api"]
condition "catalog-services" {
datacenter = "dc1"
regexp = "web.*"
source_includes_var = false
use_as_module_input = false
}
}
service {
name = "web-api"
datacenter = "dc2"
module_input "services" {
names = ["web-api"]
datacenter = "dc2"
}
}
```
The module selected for a task will likely determine whether the task should be configured with a catalog-services condition. Module authors have the option to include and use the [`catalog_services` input variable](/docs/nia/terraform-modules#catalog-services-variable) in addition to the required `services` variable. When this variable is included in the module, the user of this module will need to configure the task condition's [`source_includes_var`](/docs/nia/configuration#source_includes_var) value to be true. Catalog-Services condition modules should have the condition type and `source_includes_var` configuration value documented for users to reference when configuring a task.
Using the condition block's `use_as_module_input` field, users can configure CTS to use the condition's object as module input for the [`catalog_services` input variable](/docs/nia/terraform-modules#catalog-services-variable). Users can refer to the configured module's documentation on how to set `use_as_module_input`.
In addition to `source_includes_var`, a task's catalog-services condition has other configuration fields that can be used to specify which services' registration changes should execute the task. For example, registration changes can be filtered for services in a specific datacenter or namespace. See the [Catalog-Services Condition](/docs/nia/configuration#catalog-services-condition) configuration section for more details.
One particular condition configuration, [`regexp`](/docs/nia/configuration#regexp), filters service registration changes by services that have a name which matches the configured regular expression. When unconfigured, the `regexp` value will default to match on any service that has a name which matches the services listed in [`task.services`](/docs/nia/configuration#services). As such, either `regexp` or `task.services` must be configured. When both are configured, see example above, the task will monitor and execute on registration changes to services matching the `regexp`, and the task will additionally monitor but not execute on service instances listed in `task.services` to optionally provide additional service information to the module.
See the [Catalog-Services Condition](/docs/nia/configuration#catalog-services-condition) configuration section for further details and additional configuration options.
### Consul KV Condition
@ -146,51 +127,59 @@ Based on the `recurse` option, the condition either monitors a single Consul KV
task {
name = "consul_kv_condition_task"
description = "execute on changes to Consul KV entry"
source = "path/to/consul-kv-module"
module = "path/to/consul-kv-module"
providers = ["my-provider"]
services = ["web-api"]
condition "consul-kv" {
path = "my-key"
recurse = true
datacenter = "dc1"
namespace = "default"
source_includes_var = true
use_as_module_input = true
}
}
```
If the task condition's [`source_includes_var`](/docs/nia/configuration#source_includes_var-1) field is set to `true`, then the value of the Consul KV pair(s) will be available in the [`consul_kv` input variable](/docs/nia/terraform-modules#consul-kv-variable). To use the variable, add `consul_kv` as an input variable to the module, in addition to the required `services` variable. The condition type and `source_includes_var` configuration value should be documented in the module so that users can reference them when configuring a task.
Using the condition block's `use_as_module_input` field, users can configure CTS to use the condition's object as module input for the [`consul_kv` input variable](/docs/nia/terraform-modules#consul-kv-variable). Users can refer to the configured module's documentation on how to set `use_as_module_input`.
See the [Consul-KV Condition](/docs/nia/configuration#consul-kv-condition) configuration section for more details and additional configuration options.
### Schedule Condition
All scheduled tasks must be configured with a schedule condition. The schedule condition sets the cadence to trigger a task with a [`cron`](/docs/nia/configuration#cron) configuration. The schedule condition block does not support parameters to configure source input. As a result, inputs must be configured separately. You can configure [`task.services`](/docs/nia/configuration#services) or a [`source_input` block](/docs/nia/configuration#source_input) to set the source input.
All scheduled tasks must be configured with a schedule condition. The schedule condition sets the cadence to trigger a task with a [`cron`](/docs/nia/configuration#cron) configuration. The schedule condition block does not support parameters to configure module input. As a result, inputs must be configured separately. You can configure [`module_input` blocks](/docs/nia/configuration#module_input) to define the module inputs.
Below is an example configuration for a task that will execute every Monday, which is set by the schedule condition’s [`cron`](/docs/nia/configuration#cron) configuration. The source input is defined by the `task.services` configuration. When the task is triggered on Monday, it will retrieve the latest information on "web" and "db" from Consul and provide this to the module’s input variables.
Below is an example configuration for a task that will execute every Monday, which is set by the schedule condition's [`cron`](/docs/nia/configuration#cron) configuration. The module input is defined by the `module_input` block. When the task is triggered on Monday, it will retrieve the latest information on "web" and "db" from Consul and provide this to the module's input variables.
```hcl
task {
name = "scheduled_task"
description = "execute every Monday using service information from web and db"
services = ["web", "db"]
source = "path/to/module"
module = "path/to/module"
condition "schedule" {
cron = "* * * * Mon"
}
module_input "services" {
names = ["web", "db"]
}
}
```
Below are the available options for source input types and how to configure them:
Below are the available options for module input types and how to configure them:
- [Services source input](/docs/nia/terraform-modules/#services-source-input): configure through [`task.services`](/docs/nia/configuration#services) or [`source_input "services"`](/docs/nia/configuration#services-source-input)
- [Consul KV source input](/docs/nia/terraform-modules/#consul-kv-source-input): configure through [`source_input "consul-kv"`](/docs/nia/configuration#consul-kv-source-input)
- [Services module input](/docs/nia/terraform-modules/#services-module-input):
- [`task.services`](/docs/nia/configuration#services) field (deprecated)
- [`module_input "services"`](/docs/nia/configuration#services-configure-input) block
- Block was previously named `source_input "services"` (deprecated)
- [Consul KV module input](/docs/nia/terraform-modules/#consul-kv-module-input):
- [`module_input "consul-kv"`](/docs/nia/configuration#consul-kv-module-input)
- Block was previously named `source_input "consul-kv"` (deprecated)
#### Running Behavior
Scheduled tasks generally run on schedule, but they can be triggered on demand when running Consul-Terraform-Sync in the following ways:
Scheduled tasks generally run on schedule, but they can be triggered on demand when running CTS in the following ways:
- [Long-running mode](/docs/nia/cli#long-running-mode): At the beginning of the long-running mode, Consul-Terraform-Sync first passes through a once-mode phase in which all tasks are executed once. Scheduled tasks will trigger once during this once-mode phase. This behavior also applies to tasks that are not scheduled. After once-mode has completed, scheduled tasks subsequently trigger on schedule.
- [Long-running mode](/docs/nia/cli#long-running-mode): At the beginning of the long-running mode, CTS first passes through a once-mode phase in which all tasks are executed once. Scheduled tasks will trigger once during this once-mode phase. This behavior also applies to tasks that are not scheduled. After once-mode has completed, scheduled tasks subsequently trigger on schedule.
- [Inspect mode](/docs/nia/cli#inspect-mode): When running in inspect mode, the terminal will output a plan of proposed updates that would be made if the tasks were to trigger at that moment and then exit. No changes are applied in this mode. The outputted plan for a scheduled task is also the proposed updates that would be made if the task was triggered at that moment, even if off-schedule.
@ -208,7 +197,7 @@ Because scheduled tasks trigger on a configured cadence, buffer periods are disa
## Task Automation
Consul-Terraform-Sync will attempt to execute each enabled task once upon startup to synchronize infrastructure with the current state of Consul. The daemon will stop and exit if any error occurs while preparing the automation environment or executing a task for the first time. This helps ensure tasks have proper configuration and are executable before the daemon transitions into running tasks in full automation as service changes are discovered over time. As a result, it is not recommended to configure a task as disabled from the start. After all tasks have successfully executed once, task failures during automation will be logged and retried or attempted again after a subsequent change.
CTS will attempt to execute each enabled task once upon startup to synchronize infrastructure with the current state of Consul. The daemon will stop and exit if any error occurs while preparing the automation environment or executing a task for the first time. This helps ensure tasks have proper configuration and are executable before the daemon transitions into running tasks in full automation as service changes are discovered over time. As a result, it is not recommended to configure a task as disabled from the start. After all tasks have successfully executed once, task failures during automation will be logged and retried or attempted again after a subsequent change.
Tasks are executed near-real time when service changes are detected. For services or environments that are prone to flapping, it may be useful to configure a [buffer period](/docs/nia/configuration#buffer_period-1) for a task to accumulate changes before it is executed. The buffer period would reduce the number of consecutive network calls to infrastructure by batching changes for a task over a short duration of time.
@ -224,7 +213,7 @@ These three levels form a hierarchy where each level of data informs the one hig
### Event
When a task is triggered, Consul-Terraform-Sync takes a series of steps in order to update the network infrastructure. These steps consist of fetching the latest data from Consul for the task's source inputs and then updating the network infrastructure accordingly. An event captures information across this process. It stores information to help understand if the update to network infrastructure was successful or not and any errors that may have occurred.
When a task is triggered, CTS takes a series of steps in order to update the network infrastructure. These steps consist of fetching the latest data from Consul for the task's module inputs and then updating the network infrastructure accordingly. An event captures information across this process. It stores information to help understand if the update to network infrastructure was successful or not and any errors that may have occurred.
A dynamic task will store an event when it is triggered by a change in Consul. A scheduled task will store an event when it is triggered on schedule, regardless if there is a change in Consul. A disabled task does not update network infrastructures, so it will not store events until until re-enabled.

126
website/content/docs/nia/terraform-modules.mdx
View File

@ -7,52 +7,56 @@ description: >-
# Compatible Terraform Modules for Network Infrastructure Automation
Consul-Terraform-Sync automates execution of Terraform modules through tasks. A task is a construct in Consul-Terraform-Sync that defines the automation of Terraform and the module.
Consul-Terraform-Sync (CTS) automates execution of Terraform modules through tasks. A task is a construct in CTS that defines the automation of Terraform and the module.
## Module Specifications
Compatible modules for Consul-Terraform-Sync follow the [standard module](https://www.terraform.io/docs/modules/index.html#standard-module-structure) structure. Modules can use syntax supported by Terraform version 0.13 and newer.
Compatible modules for CTS follow the [standard module](https://www.terraform.io/docs/modules/index.html#standard-module-structure) structure. Modules can use syntax supported by Terraform version 0.13 and newer.
### Compatibility Requirements
Below are the two required elements for module compatibility with Consul-Terraform-Sync
Below are the two required elements for module compatibility with CTS
1. **Root module** - Terraform has one requirement for files in the root directory of the repository to function as the primary entrypoint for the module. It should encapsulate the core logic to be used by Consul-Terraform-Sync for task automation. `main.tf` is the recommended filename for the main file where resources are created.
2. [**`services` input variable**](#services-variable) - Consul-Terraform-Sync requires all modules to have the following input variable declared within the root module. The declaration of the `services` variable can be included at the top of the suggested `variables.tf` file where other input variables are commonly declared. This variable functions as the response object from the Consul catalog API and surfaces network information to be consumed by the module. It is structured as a map of objects.
1. **Root module** - Terraform has one requirement for files in the root directory of the repository to function as the primary entrypoint for the module. It should encapsulate the core logic to be used by CTS for task automation. `main.tf` is the recommended filename for the main file where resources are created.
2. [**`services` input variable**](#services-variable) - CTS requires all modules to have the following input variable declared within the root module. The declaration of the `services` variable can be included at the top of the suggested `variables.tf` file where other input variables are commonly declared. This variable functions as the response object from the Consul catalog API and surfaces network information to be consumed by the module. It is structured as a map of objects.
### Optional Input Variables
In addition to the required `services` input variable, Consul-Terraform-Sync provides additional, optional input variables to be used within your module. Support for an optional input variable requires two changes:
In addition to the required `services` input variable, CTS provides additional, optional input variables to be used within your module. Support for an optional input variable requires two changes:
1. Updating the Terraform Module to declare the input variable in the suggested `variables.tf`
1. Adding configuration to the Consul-Terraform-Sync task block to define the source input values that should be provided to the input variables
1. Adding configuration to the CTS task block to define the module input values that should be provided to the input variables
See below sections for more information on [defining source input](#source-input) and [declaring optional input variables](#how-to-create-a-compatible-terraform-module) in your Terraform module.
See below sections for more information on [defining module input](#module-input) and [declaring optional input variables](#how-to-create-a-compatible-terraform-module) in your Terraform module.
### Source Input
### Module Input ((#source-input))
A source input allows for a task to satisfy the input requirements defined by the Terraform Module’s [input variables](https://www.terraform.io/docs/language/values/variables.html), and is configured alongside a task’s condition. Both the source input and condition define objects to be monitored, but for differing reasons.
A task monitors [Consul objects](/docs/nia#consul-objects) that are defined by the task's configuration. The Consul objects can be used for the module input that satisfies the requirements defined by the task's Terraform module's [input variables](https://www.terraform.io/docs/language/values/variables.html).
The condition defines monitored objects with criteria. When this criteria is satisfied, Consul-Terraform-Sync will then trigger a task.
A task's module input is slightly different from the task's condition, even though both monitor defined objects. The task's condition monitors defined objects with a configured criteria. When this criteria is satisfied, the task will trigger.
The source input however, defines monitored objects with the intent of providing values or metadata about these objects to the Terraform Module. The source input and condition objects can be the same, such as when `task.services` is provided without a source input block and condition block, but they can also be defined using separate blocks. In this way the source input does not need to be tied to the provided condition in order to satisfy the Terraform Module.
The module input, however, monitors defined objects with the intent of providing values or metadata about these objects to the Terraform module. The monitored module input and condition objects can be the same object, such as a task configured with a `condition "services"` block and `use_as_module_input` set to `true`. The module input and condition can also be different objects and configured separately, such as a task configured with a `condition "catalog-services` and `module_input "consul-kv"` block. As a result, the monitored module input is decoupled from the provided condition in order to satisfy the Terraform module.
There are a few ways that a source input can be defined:
Each type of object that CTS monitors can only be defined through one configuration within a task definition. For example, if a task monitors services, the task cannot have both `condition "services"` and `module_input "services"` configured. See [Task Module Input configuration](/docs/nia/configuration#task-module-input) for more details.
- [**`services` list**](/docs/nia/configuration#services) - The list of services to act as a source input.
- **`source_includes_var` condition field** - If the condition supports this field, and it is set to true, then the condition’s objects will be used as a source input. For example, if a module is defined supporting [`catalog_services` input variable](#catalog-services-variable) then, this field can be set to true in the [catalog-services condition](/docs/nia/tasks#catalog-services-condition).
- [**`source_input` block**](/docs/nia/configuration#source-input) - This block specifically defines a source input.
There are a few ways that a module input can be defined:
Multiple ways of defining a source input adds configuration flexibility, and allows for optional additional input variables to be supported by Consul-Terraform-Sync alongside the `services` input variable.
- [**`services` list**](/docs/nia/configuration#services) (deprecated) - The list of services to use as module input.
- **`condition` block's `use_as_module_input` field** - When set to true, the condition's objects are used as module input.
- Field was previously named `source_includes_var` (deprecated)
- [**`module_input` blocks**](/docs/nia/configuration#module-input) - This block can be configured multiple times to define objects to use as module input.
- Block was previously named `source_input` (deprecated)
These optional input variables include:
Multiple ways of defining a module input adds configuration flexibility, and allows for optional additional input variables to be supported by CTS alongside the `services` input variable.
Additional optional input variable types:
- [**`catalog_services` variable**](#catalog-services-variable)
- [**`consul_kv` variable**](#consul-kv-variable)
#### Services Source Input
#### Services Module Input ((#services-source-input))
Tasks configured with a services source input monitor for changes to services. Monitoring is either performed on a configured list of services or on any services matching a provided regex.
Tasks configured with a services module input monitor for changes to services. Monitoring is either performed on a configured list of services or on any services matching a provided regex.
Sample rendered services input:
@ -89,13 +93,15 @@ services = {
</CodeBlockConfig>
In order to configure a task with the services source input, the list of services that will be used for the input must be configured in one of the following ways:
In order to configure a task with the services module input, the list of services that will be used for the input must be configured in one of the following ways:
- the task's [`services`](/docs/nia/configuration#services)
- a [`condition "services"` block](/docs/nia/configuration#services-condition) configured with `regexp` and `source_includes_var` set to true
- a [`source_input "services"` block](/docs/nia/configuration#services-source-input) configured with `regexp`
- the task's [`services`](/docs/nia/configuration#services) (deprecated)
- a [`condition "services"` block](/docs/nia/configuration#services-condition) configured with `use_as_module_input` field set to true
- Field was previously named `source_includes_var` (deprecated)
- a [`module_input "services"` block](/docs/nia/configuration#services-module-input)
- Block was previously named `source_input "services"` (deprecated)
The services source input operates by monitoring the [Health List Nodes For Service API](/api-docs/health#list-nodes-for-service) and provides the latest service information to the input variables. A complete list of service information that would be provided to the module is expanded below:
The services module input operates by monitoring the [Health List Nodes For Service API](/api-docs/health#list-nodes-for-service) and provides the latest service information to the input variables. A complete list of service information that would be provided to the module is expanded below:
| Attribute | Description |
| ----------------------- | ------------------------------------------------------------------------------------------------- |
@ -114,26 +120,26 @@ The services source input operates by monitoring the [Health List Nodes For Serv
| `node_tagged_addresses` | List of explicit LAN and WAN IP addresses for the agent |
| `node_meta` | List of user-defined metadata key/value pairs for the node |
Below is an example configuration for a task that will execute on a schedule and provide information about the services matching the `regexp` parameter to the task’s module. Note that because `regexp` is set, `task.services` is omitted, and since a schedule is being used to trigger task execution, a `condition "services"` block cannot be used.
Below is an example configuration for a task that will execute on a schedule and provide information about the services matching the `regexp` parameter to the task’s module.
```hcl
task {
name = "services_condition_task"
description = "execute on changes to services whose name starts with web"
providers = ["my-provider"]
source = "path/to/services-condition-module"
module = "path/to/services-condition-module"
condition "schedule" {
cron = "* * * * Mon"
}
source_input "services" {
module_input "services" {
regexp = "^web.*"
}
}
```
#### Consul KV Source Input
#### Consul KV Module Input ((#consul-kv-source-input))
Tasks configured with a Consul KV source input monitor Consul KV for changes to KV pairs that satisfy the provided configuration. The Consul KV source input operates by monitoring the [Consul KV API](/api-docs/kv#read-key) and provides these key values to the task’s module.
Tasks configured with a Consul KV module input monitor Consul KV for changes to KV pairs that satisfy the provided configuration. The Consul KV module input operates by monitoring the [Consul KV API](/api-docs/kv#read-key) and provides these key values to the task's module.
Sample rendered consul KV input:
@ -147,26 +153,26 @@ consul_kv = {
</CodeBlockConfig>
To configure a task with the Consul KV source input, the KVs which will be used for the input must be configured in one of the following ways:
To configure a task with the Consul KV module input, the KVs which will be used for the input must be configured in one of the following ways:
- a [`condition "consul-kv"` block](/docs/nia/configuration#consul-kv-condition) configured with the `source_includes_var` set to true.
- a [`source_input "consul-kv"` block](/docs/nia/configuration#consul-kv-source-input).
- a [`condition "consul-kv"` block](/docs/nia/configuration#consul-kv-condition) configured with the `use_as_module_input` field set to true.
- Field was previously named `source_includes_var` (deprecated)
- a [`module_input "consul-kv"` block](/docs/nia/configuration#consul-kv-module-input).
- Block was previously named `source_input "consul-kv"` (deprecated)
Below is a similar example to the one provided in the [Consul KV Condition](/docs/nia/tasks#consul-kv-condition) section. However, the difference in this example is that instead of triggering based on a change to Consul KV, this task will instead execute on a schedule. Once execution is triggered, Consul KV information is then provided to the task’s module.
Below is a similar example to the one provided in the [Consul KV Condition](/docs/nia/tasks#consul-kv-condition) section. However, the difference in this example is that instead of triggering based on a change to Consul KV, this task will instead execute on a schedule. Once execution is triggered, Consul KV information is then provided to the task's module.
```hcl
task {
name = "consul_kv_schedule_task"
description = "executes on Monday monitoring Consul KV"
providers = ["my-provider"]
services = ["web-api"]
source = "path/to/consul-kv-module"
module = "path/to/consul-kv-module"
condition "schedule" {
cron = "* * * * Mon"
}
source_input "consul-kv" {
module_input "consul-kv" {
path = "my-key"
recurse = true
datacenter = "dc1"
@ -175,9 +181,9 @@ task {
}
```
#### Catalog Services Source Input
#### Catalog Services Module Input ((#catalog-services-source-input))
Tasks configured with a Catalog Services source input monitors for service and tag information provided by the [Catalog List Services API](/api-docs/catalog#list-services). The source input is a map of service names to a list of tags.
Tasks configured with a Catalog Services module input monitors for service and tag information provided by the [Catalog List Services API](/api-docs/catalog#list-services). The module input is a map of service names to a list of tags.
Sample rendered catalog-services input:
@ -193,26 +199,26 @@ catalog_services = {
</CodeBlockConfig>
To configure a task with the Catalog Services source input, the catalog services which will be used for the input must be configured in one of the following ways:
To configure a task with the Catalog Services module input, the catalog services which will be used for the input must be configured in one of the following ways:
- a [`condition "catalog-services"` block](/docs/nia/configuration#consul-kv-condition) configured with `source_includes_var` set to true.
- a [`condition "catalog-services"` block](/docs/nia/configuration#consul-kv-condition) configured with `use_as_module_input` field.
- Field was previously named `source_includes_var` (deprecated)
-> **Note:** Currently there is no support for a `source_input “catalog-services”` block.
-> **Note:** Currently there is no support for a `module_input "catalog-services"` block.
Example of a catalog-services condition which supports source input through `source_includes_var`:
Example of a catalog-services condition which supports module input through `use_as_module_input`:
```hcl
task {
name = "catalog_services_condition_task"
description = "execute on registration/deregistration of services"
providers = ["my-provider"]
services = ["web-api"]
source = "path/to/catalog-services-module"
module = "path/to/catalog-services-module"
condition "catalog-services" {
datacenter = "dc1"
namespace = "default"
regexp = "web.*"
source_includes_var = true
use_as_module_input = true
node_meta {
key = "value"
}
@ -222,13 +228,13 @@ task {
## How to Create a Compatible Terraform Module
You can read more on how to [create a module](https://www.terraform.io/docs/modules/index.html) or work through a [tutorial to build a module](https://learn.hashicorp.com/tutorials/terraform/module-create). Consul-Terraform-Sync is designed to integrate with any module that satisfies the specifications in the following section.
You can read more on how to [create a module](https://www.terraform.io/docs/modules/index.html) or work through a [tutorial to build a module](https://learn.hashicorp.com/tutorials/terraform/module-create). CTS is designed to integrate with any module that satisfies the specifications in the following section.
The repository [hashicorp/consul-terraform-sync-template-module](https://github.com/hashicorp/consul-terraform-sync-template-module) can be cloned and used as a starting point for structuring a compatible Terraform module. The template repository has the files described in the next steps prepared.
First, create a directory to organize Terraform configuration files that make up the module. You can start off with creating two files `main.tf` and `variables.tf` and expand from there based on your module and network infrastructure automation needs.
The `main.tf` is the entry point of the module and this is where you can begin authoring your module. It can contain multiple Terraform resources related to an automation task that uses Consul service discovery information, particularly the required [`services` input variable](#services-variable). The code example below shows a resource using the `services` variable. When this example is used in automation with Consul-Terraform-Sync, the content of the local file would dynamically update as Consul service discovery information changes.
The `main.tf` is the entry point of the module and this is where you can begin authoring your module. It can contain multiple Terraform resources related to an automation task that uses Consul service discovery information, particularly the required [`services` input variable](#services-variable). The code example below shows a resource using the `services` variable. When this example is used in automation with CTS, the content of the local file would dynamically update as Consul service discovery information changes.
<CodeBlockConfig filename="main.tf">
@ -244,11 +250,11 @@ resource "local_file" "consul_services" {
</CodeBlockConfig>
Something important to consider before authoring your module is deciding the [condition under which it will execute](/docs/nia/tasks#task-execution). This will allow you to potentially include other types of Consul-Terraform-Sync provided input variables in your module. It will also help inform your documentation and how users should configure their task for your module.
Something important to consider before authoring your module is deciding the [condition under which it will execute](/docs/nia/tasks#task-execution). This will allow you to potentially use other types of CTS provided input variables in your module. It will also help inform your documentation and how users should configure their task for your module.
### Services Variable
To satisfy the specification requirements for a compatible module, copy the `services` variable declaration to the `variables.tf` file. Your module can optionally have other [variable declarations](#module-input-variables) and [Consul-Terraform-Sync provided input variables](/docs/nia/terraform-modules#optional-input-variables) in addition to `var.services`.
To satisfy the specification requirements for a compatible module, copy the `services` variable declaration to the `variables.tf` file. Your module can optionally have other [variable declarations](#module-input-variables) and [CTS provided input variables](/docs/nia/terraform-modules#optional-input-variables) in addition to `var.services`.
<CodeBlockConfig filename="variables.tf">
@ -282,9 +288,9 @@ variable "services" {
</CodeBlockConfig>
Keys of the `services` map are unique identifiers of the service across Consul agents and data centers. Keys follow the format `service-id.node.datacenter` (or `service-id.node.namespace.datacenter` for Consul Enterprise). A complete list of attributes available for the `services` variable is included in the [documentation for Consul-Terraform-Sync tasks](/docs/nia/tasks#services-condition).
Keys of the `services` map are unique identifiers of the service across Consul agents and data centers. Keys follow the format `service-id.node.datacenter` (or `service-id.node.namespace.datacenter` for Consul Enterprise). A complete list of attributes available for the `services` variable is included in the [documentation for CTS tasks](/docs/nia/tasks#services-condition).
Terraform variables when passed as module arguments can be [lossy for object types](https://www.terraform.io/docs/configuration/types.html#conversion-of-complex-types). This allows Consul-Terraform-Sync to declare the full variable with every object attribute in the generated root module, and pass the variable to a child module that contains a subset of these attributes for its variable declaration. Modules compatible with Consul-Terraform-Sync may simplify the `var.services` declaration within the module by omitting unused attributes. For example, the following services variable has 4 attributes with the rest omitted.
Terraform variables when passed as module arguments can be [lossy for object types](https://www.terraform.io/docs/configuration/types.html#conversion-of-complex-types). This allows CTS to declare the full variable with every object attribute in the generated root module, and pass the variable to a child module that contains a subset of these attributes for its variable declaration. Modules compatible with CTS may simplify the `var.services` declaration within the module by omitting unused attributes. For example, the following services variable has 4 attributes with the rest omitted.
<CodeBlockConfig filename="variables.tf">
@ -324,7 +330,7 @@ The keys of the `catalog_services` map are the names of the services that are re
We recommend that if you make a module with with a catalog-services condition, that you document this in the README. This way, users that want to configure a task with your module will know to configure a catalog-services [condition](/docs/nia/configuration#condition) block.
Similarly, if you include the `catalog_services` variable in your module, we recommend that you also document this usage in the README. Users of your module will then know to set the catalog-services condition [`source_includes_var`](/docs/nia/configuration#source_includes_var) configuration to be true. When this field is set to true, Consul-Terraform-Sync will declare the `catalog_services` variable in the generated root module, and pass the variable to a child module. Therefore, if this field is configured inconsistently, Consul-Terraform-Sync will error and exit.
Similarly, if you use the `catalog_services` variable in your module, we recommend that you also document this usage in the README. Users of your module will then know to set the catalog-services condition [`use_as_module_input`](/docs/nia/configuration#catalog-services-condition) configuration to be true. When this field is set to true, CTS will declare the `catalog_services` variable in the generated root module, and pass the variable to a child module. Therefore, if this field is configured inconsistently, CTS will error and exit.
### Consul KV Variable
@ -341,7 +347,7 @@ variable "consul_kv" {
</CodeBlockConfig>
If your module contains the `consul_kv` variable, we recommend documenting the usage in the README file so that users know to set the [`source_includes_var`](/docs/nia/configuration#source_includes_var-1) configuration to `true` in the `consul-kv` condition. Setting the field to `true` instructs Consul-Terraform-Sync to declare the `consul_kv` variable in the generated root module and pass the variable to a child module. Therefore, if this field is configured inconsistently, Consul-Terraform-Sync will error and exit.
If your module contains the `consul_kv` variable, we recommend documenting the usage in the README file so that users know to set the [`use_as_module_input`](/docs/nia/configuration#consul-kv-condition) configuration to `true` in the `consul-kv` condition. Setting the field to `true` instructs CTS to declare the `consul_kv` variable in the generated root module and pass the variable to a child module. Therefore, if this field is configured inconsistently, CTS will error and exit.
### Module Input Variables
@ -353,15 +359,15 @@ Network infrastructure differs vastly across teams and organizations, and the au
4. Set reasonable default values for variables that are optional, or omit default values for variables that are required module arguments.
5. Set the [sensitive argument](https://www.terraform.io/docs/language/values/variables.html#suppressing-values-in-cli-output) for variables that contain secret or sensitive values. When set, Terraform will redact the value from output when Terraform commands are run.
Terraform is an explicit configuration language and requires variables to be declared, typed, and passed explicitly through as module arguments. Consul-Terraform-Sync abstracts this by creating intermediate variables at the root level from the source input. These values are configured by practitioners within the [`task` block](/docs/nia/configuration#variable_files). Value assignments are parsed to interpolate the corresponding variable declaration and are written to the appropriate Terraform files. A few assumptions are made for the intermediate variables: the variables users provide Consul-Terraform-Sync are declared and supported by the module, matching name and type.
Terraform is an explicit configuration language and requires variables to be declared, typed, and passed explicitly through as module arguments. CTS abstracts this by creating intermediate variables at the root level from the module input. These values are configured by practitioners within the [`task` block](/docs/nia/configuration#variable_files). Value assignments are parsed to interpolate the corresponding variable declaration and are written to the appropriate Terraform files. A few assumptions are made for the intermediate variables: the variables users provide CTS are declared and supported by the module, matching name and type.
### Module Guidelines
This section covers guidelines for authoring compatible Consul-Terraform-Sync modules.
This section covers guidelines for authoring compatible CTS modules.
#### Scope
We recommend scoping the module to a few related resources for a provider. Small modules are easier and more flexible for end users to adopt for Consul-Terraform-Sync. It allows them to iteratively combine different modules and use them as building blocks to meet their unique network infrastructure needs.
We recommend scoping the module to a few related resources for a provider. Small modules are easier and more flexible for end users to adopt for CTS. It allows them to iteratively combine different modules and use them as building blocks to meet their unique network infrastructure needs.
#### Complexity
@ -371,8 +377,8 @@ Consider authoring modules with low complexity to reduce the run time for Terraf
The Terraform module must declare which providers it requires within the [`terraform.required_providers` block](https://www.terraform.io/docs/language/providers/requirements.html#requiring-providers). We suggest to also include a version constraint for the provider to specify which versions the module is compatible with.
Aside from the `required_providers` block, provider configurations should not be included within the sharable module for network integrations. End users will configure the providers through Consul-Terraform-Sync, and Consul-Terraform-Sync will then translate provider configuration to the generated root module appropriately.
Aside from the `required_providers` block, provider configurations should not be included within the sharable module for network integrations. End users will configure the providers through CTS, and CTS will then translate provider configuration to the generated root module appropriately.
#### Documentation
Modules for Consul-Terraform-Sync are Terraform modules and can effectively run independently from the `consul-terraform-sync` daemon and Consul environment. They should be written and designed with Terraform best practices and should be clear to a Terraform user what the module does and how to use it. Module documentation should be named `README` or `README.md`. The description should capture what the module should be used for and the implications of running it in automation with Consul-Terraform-Sync.
Modules for CTS are Terraform modules and can effectively run independently from the `consul-terraform-sync` daemon and Consul environment. They should be written and designed with Terraform best practices and should be clear to a Terraform user what the module does and how to use it. Module documentation should be named `README` or `README.md`. The description should capture what the module should be used for and the implications of running it in automation with CTS.

9
website/data/docs-nav-data.json
View File

@ -793,6 +793,15 @@
{
"title": "Compatibility",
"path": "nia/compatibility"
},
{
"title": "Release Notes",
"routes": [
{
"title": "v0.5.0",
"path": "nia/release-notes/0-5-0"
}
]
}
]
},