5b53b5aef5
* website: migrate to new nav-data format * website: clean up unused intro content * website: remove deprecated sidebar_title from frontmatter * website: add react-content to fix global style import issue
127 lines
8.7 KiB
Plaintext
127 lines
8.7 KiB
Plaintext
---
|
|
layout: docs
|
|
page_title: Tasks
|
|
description: >-
|
|
Consul-Terraform-Sync Tasks
|
|
---
|
|
|
|
# Tasks
|
|
|
|
A task is the translation of dynamic service information from the Consul Catalog into network infrastructure changes downstream. Consul-Terraform-Sync carries out automation for executing tasks using network drivers. For a Terraform driver, the scope of a task is a Terraform module.
|
|
|
|
Below is an example task configuration:
|
|
|
|
```hcl
|
|
task {
|
|
name = "frontend-firewall-policies"
|
|
description = "Add firewall policy rules for frontend services"
|
|
providers = ["fake-firewall", "null"]
|
|
services = ["web", "image"]
|
|
source = "example/firewall-policy/module"
|
|
version = "1.0.0"
|
|
}
|
|
```
|
|
|
|
In the example task above, the "fake-firewall" and "null" providers, listed in the `providers` field, are used. These providers themselves should be configured in their own separate [`terraform_provider` blocks](/docs/nia/configuration#terraform-provider). These providers are used in the Terraform module "example/firewall-policy/module", configured in the `source` field, to create, update, and destroy resources. This module may do something like use the providers to create and destroy firewall policy objects based on IP addresses. The IP addresses come from the "web" and "image" service instances configured in the `services` field. This service-level information is retrieved by Consul-Terraform-Sync which watches Consul catalog for changes.
|
|
|
|
See [task configuration](/docs/nia/configuration#task) for more details on how to configure a task.
|
|
|
|
A task can be either enabled or disabled using the [task cli](/docs/nia/cli/task). When enabled, tasks are executed and automated as described in sections below. However, disabled tasks do not execute when changes are detected from Consul catalog. Since disabled tasks do not execute, they also do not store [events](/docs/nia/tasks#event) until re-enabled.
|
|
|
|
## Task Execution
|
|
|
|
An enabled task is executed when any change of information for services the task is configured for is detected from the Consul catalog. Execution could include one or more changes to service values, like IP address, added or removed service instance, or tags. A complete list of values that would cause a task to run are expanded below:
|
|
|
|
| Attribute | Description |
|
|
| ----------------------- | ------------------------------------------------------------------------------------------------- |
|
|
| `id` | A unique Consul ID for this service. This is unique per Consul agent. |
|
|
| `name` | The logical name of the service. Many service instances may share the same logical service name. |
|
|
| `address` | IP address of the service host -- if empty, node address should be used. |
|
|
| `port` | Port number of the service |
|
|
| `meta` | List of user-defined metadata key/value pairs for the service |
|
|
| `tags` | List of tags for the service |
|
|
| `namespace` | Consul Enterprise namespace of the service instance |
|
|
| `status` | Representative status for the service instance based on an aggregate of the list of health checks |
|
|
| `node` | Name of the Consul node on which the service is registered |
|
|
| `node_id` | ID of the node on which the service is registered. |
|
|
| `node_address` | The IP address of the Consul node on which the service is registered. |
|
|
| `node_datacenter` | Data center of the Consul node on which the service is registered. |
|
|
| `node_tagged_addresses` | List of explicit LAN and WAN IP addresses for the agent |
|
|
| `node_meta` | List of user-defined metadata key/value pairs for the node |
|
|
|
|
Consul-Terraform-Sync automatically generates any files needed to execute the network driver for each task. See [network drivers](/docs/nia/network-drivers) for more details on the files generated for the Terraform driver.
|
|
|
|
## Task Automation
|
|
|
|
Consul-Terraform-Sync will attempt to execute each enabled task once upon startup to synchronize infrastructure with the current state of Consul. The daemon will stop and exit if any error occurs while preparing the automation environment or executing a task for the first time. This helps ensure tasks have proper configuration and are executable before the daemon transitions into running tasks in full automation as service changes are discovered over time. As a result, it is not recommended to configure a task as disabled from the start. After all tasks have successfully executed once, task failures during automation will be logged and retried or attempted again after a subsequent change.
|
|
|
|
Tasks are executed near-real time when service changes are detected. For services or environments that are prone to flapping, it may be useful to configure a [buffer period](/docs/nia/configuration#buffer_period-1) for a task to accumulate changes before it is executed. The buffer period would reduce the number of consecutive network calls to infrastructure by batching changes for a task over a short duration of time.
|
|
|
|
## Status Information
|
|
|
|
Status-related information is collected and offered via [status API](/docs/nia/api#status) to provide visibility into what and how the tasks are running. Information is offered in three-levels (lowest to highest):
|
|
- Event data
|
|
- Task status
|
|
- Overall status
|
|
|
|
These three levels form a hierarchy where each level of data informs the one higher. The lowest-level, event data, is collected each time a task runs to update network infrastructure. This event data is then aggregated to inform individual task statuses. The count distribution of all the task statuses inform the overall status's task summary.
|
|
|
|
### Event
|
|
|
|
Each time a task's services has an update, Consul-Terraform-Sync takes a series of steps in order to update network infrastructure. This process starts with updating the task's templates to fetch new service data from Consul and ends with any post-actions after modifying network infrastructure. An event is a data structure that captures information on this process of updating network infrastructure. It stores information to help understand if the update to network infrastructure was successful or not, and it stores any errors that occurred. Because disabled tasks do not update network infrastructures, they therefore do not have store events until re-enabled.
|
|
|
|
Sample event:
|
|
```json
|
|
{
|
|
"id": "ef202675-502f-431f-b133-ed64d15b0e0e",
|
|
"success": false,
|
|
"start_time": "2020-11-24T12:05:18.651231-05:00",
|
|
"end_time": "2020-11-24T12:05:20.900115-05:00",
|
|
"task_name": "task_b",
|
|
"error": {
|
|
"message": "example error: error while doing terraform-apply"
|
|
},
|
|
...
|
|
}
|
|
```
|
|
|
|
For complete information on the event structure, see [events in our API documentation](/docs/nia/api#event). Event information can be retrieved by using the [`include=events` parameter](/docs/nia/api#include) with the [task status API](/docs/nia/api#task-status).
|
|
|
|
### Task Status
|
|
|
|
Each time a task runs to update network infrastructure, event data is stored for that run. 5 most recent events are stored for each task, and these stored events are used to determine task status. For example, if the most recent stored event is not successful but the others are, then the task's health status is "errored".
|
|
|
|
Sample task status:
|
|
```json
|
|
{
|
|
"task_name": "task_b",
|
|
"status": "errored",
|
|
"providers": [
|
|
"null"
|
|
],
|
|
"services": [
|
|
"web",
|
|
],
|
|
"events_url": "/v1/status/tasks/task_b?include=events",
|
|
}
|
|
```
|
|
|
|
Task status information can be retrieved with [task status API](/docs/nia/api#task-status). The API documentation includes details on what health statuses are available and how it is calculated based on events' success/failure information.
|
|
|
|
### Overall Status
|
|
|
|
Overall status returns a summary of the health statuses across all tasks. The summary is the count of tasks in each health status category.
|
|
|
|
Sample overall status:
|
|
```json
|
|
{
|
|
"task_summary": {
|
|
"successful": 28,
|
|
"errored": 5,
|
|
"critical": 1
|
|
}
|
|
}
|
|
```
|
|
|
|
Overall status information can be retrieved with [overall status API](/docs/nia/api#overall-status). The API documentation includes details on what health statuses are available and how it is calculated based on task statuses' health status information.
|