open-consul/website/source/docs/agent/options.html.md

layout	page_title	sidebar_current	description
docs	Configuration	docs-agent-config	The agent has various configuration options that can be specified via the command-line or via configuration files. All of the configuration options are completely optional. Defaults are specified with their descriptions.

Configuration

The agent has various configuration options that can be specified via the command-line or via configuration files. All of the configuration options are completely optional. Defaults are specified with their descriptions.

Configuration precedence is evaluated in the following order:

1. Command line arguments
2. Environment Variables
3. Configuration files

When loading configuration, Consul loads the configuration from files and directories in lexical order. For example, configuration file basic_config.json will be processed before extra_config.json. Configuration can be in either HCL or JSON format. Available in Consul 1.0 and later, the HCL support now requires an .hcl or .json extension on all configuration files in order to specify their format.

Configuration specified later will be merged into configuration specified earlier. In most cases, "merge" means that the later version will override the earlier. In some cases, such as event handlers, merging appends the handlers to the existing configuration. The exact merging behavior is specified for each option below.

Consul also supports reloading configuration when it receives the SIGHUP signal. Not all changes are respected, but those that are documented below in the Reloadable Configuration section. The reload command can also be used to trigger a configuration reload.

You can test the following configuration options by following the Getting Started guides to install a local agent.

Command-line Options

The options below are all specified on the command-line.

• -advertise - The advertise address is used to change the address that we advertise to other nodes in the cluster. By default, the -bind address is advertised. However, in some cases, there may be a routable address that cannot be bound. This flag enables gossiping a different address to support this. If this address is not routable, the node will be in a constant flapping state as other nodes will treat the non-routability as a failure. In Consul 1.0 and later this can be set to a go-sockaddr template.

• -advertise-wan - The advertise WAN address is used to change the address that we advertise to server nodes joining through the WAN. This can also be set on client agents when used in combination with the translate_wan_addrs configuration option. By default, the -advertise address is advertised. However, in some cases all members of all datacenters cannot be on the same physical or virtual network, especially on hybrid setups mixing cloud and private datacenters. This flag enables server nodes gossiping through the public network for the WAN while using private VLANs for gossiping to each other and their client agents, and it allows client agents to be reached at this address when being accessed from a remote datacenter if the remote datacenter is configured with translate_wan_addrs. In Consul 1.0 and later this can be set to a go-sockaddr template

• -bootstrap - This flag is used to control if a server is in "bootstrap" mode. It is important that no more than one server per datacenter be running in this mode. Technically, a server in bootstrap mode is allowed to self-elect as the Raft leader. It is important that only a single node is in this mode; otherwise, consistency cannot be guaranteed as multiple nodes are able to self-elect. It is not recommended to use this flag after a cluster has been bootstrapped.

• -bootstrap-expect - This flag provides the number of expected servers in the datacenter. Either this value should not be provided or the value must agree with other servers in the cluster. When provided, Consul waits until the specified number of servers are available and then bootstraps the cluster. This allows an initial leader to be elected automatically. This cannot be used in conjunction with the legacy -bootstrap flag. This flag requires -server mode.

• -bind - The address that should be bound to for internal cluster communications. This is an IP address that should be reachable by all other nodes in the cluster. By default, this is "0.0.0.0", meaning Consul will bind to all addresses on the local machine and will advertise the private IPv4 address to the rest of the cluster. If there are multiple private IPv4 addresses available, Consul will exit with an error at startup. If you specify "[::]", Consul will advertise the public IPv6 address. If there are multiple public IPv6 addresses available, Consul will exit with an error at startup. Consul uses both TCP and UDP and the same port for both. If you have any firewalls, be sure to allow both protocols. In Consul 1.0 and later this can be set to a go-sockaddr template that needs to resolve to a single address. Some example templates:

  # Using address within a specific CIDR
  $ consul agent -bind '{{ GetPrivateInterfaces | include "network" "10.0.0.0/8" | attr "address" }}'
  

  # Using a static network interface name
  $ consul agent -bind '{{ GetInterfaceIP "eth0" }}'
  

  # Using regular expression matching for network interface name that is forwardable and up
  $ consul agent -bind '{{ GetAllInterfaces | include "name" "^eth" | include "flags" "forwardable|up" | attr "address" }}'
  

• -serf-wan-bind - The address that should be bound to for Serf WAN gossip communications. By default, the value follows the same rules as -bind command-line flag, and if this is not specified, the -bind option is used. This is available in Consul 0.7.1 and later. In Consul 1.0 and later this can be set to a go-sockaddr template

• -serf-lan-bind - The address that should be bound to for Serf LAN gossip communications. This is an IP address that should be reachable by all other LAN nodes in the cluster. By default, the value follows the same rules as -bind command-line flag, and if this is not specified, the -bind option is used. This is available in Consul 0.7.1 and later. In Consul 1.0 and later this can be set to a go-sockaddr template

• -check_output_max_size - Override the default limit of 4k for maximum size of checks, this is a positive value. By limiting this size, it allows to put less pressure on Consul servers when many checks are having a very large output in their checks. In order to completely disable check output capture, it is possible to use discard_check_output.

• -client - The address to which Consul will bind client interfaces, including the HTTP and DNS servers. By default, this is "127.0.0.1", allowing only loopback connections. In Consul 1.0 and later this can be set to a space-separated list of addresses to bind to, or a go-sockaddr template that can potentially resolve to multiple addresses.

• -config-file - A configuration file to load. For more information on the format of this file, read the Configuration Files section. This option can be specified multiple times to load multiple configuration files. If it is specified multiple times, configuration files loaded later will merge with configuration files loaded earlier. During a config merge, single-value keys (string, int, bool) will simply have their values replaced while list types will be appended together.

• -config-dir - A directory of configuration files to load. Consul will load all files in this directory with the suffix ".json" or ".hcl". The load order is alphabetical, and the the same merge routine is used as with the config-file option above. This option can be specified multiple times to load multiple directories. Sub-directories of the config directory are not loaded. For more information on the format of the configuration files, see the Configuration Files section.

• -config-format - The format of the configuration files to load. Normally, Consul detects the format of the config files from the ".json" or ".hcl" extension. Setting this option to either "json" or "hcl" forces Consul to interpret any file with or without extension to be interpreted in that format.

• -data-dir - This flag provides a data directory for the agent to store state. This is required for all agents. The directory should be durable across reboots. This is especially critical for agents that are running in server mode as they must be able to persist cluster state. Additionally, the directory must support the use of filesystem locking, meaning some types of mounted folders (e.g. VirtualBox shared folders) may not be suitable. Note: both server and non-server agents may store ACL tokens in the state in this directory so read access may grant access to any tokens on servers and to any tokens used during service registration on non-servers. On Unix-based platforms the files are written with 0600 permissions so you should ensure only trusted processes can execute as the same user as Consul. On Windows, you should ensure the directory has suitable permissions configured as these will be inherited.

• -datacenter - This flag controls the datacenter in which the agent is running. If not provided, it defaults to "dc1". Consul has first-class support for multiple datacenters, but it relies on proper configuration. Nodes in the same datacenter should be on a single LAN.

• -dev - Enable development server mode. This is useful for quickly starting a Consul agent with all persistence options turned off, enabling an in-memory server which can be used for rapid prototyping or developing against the API. In this mode, Connect is enabled and will by default create a new root CA certificate on startup. This mode is not intended for production use as it does not write any data to disk. The gRPC port is also defaulted to 8502 in this mode.

• -disable-host-node-id - Setting this to true will prevent Consul from using information from the host to generate a deterministic node ID, and will instead generate a random node ID which will be persisted in the data directory. This is useful when running multiple Consul agents on the same host for testing. This defaults to false in Consul prior to version 0.8.5 and in 0.8.5 and later defaults to true, so you must opt-in for host-based IDs. Host-based IDs are generated using https://github.com/shirou/gopsutil/tree/master/host, which is shared with HashiCorp's Nomad, so if you opt-in to host-based IDs then Consul and Nomad will use information on the host to automatically assign the same ID in both systems.

• -disable-keyring-file - If set, the keyring will not be persisted to a file. Any installed keys will be lost on shutdown, and only the given -encrypt key will be available on startup. This defaults to false.

• -dns-port - the DNS port to listen on. This overrides the default port 8600. This is available in Consul 0.7 and later.

• -domain - By default, Consul responds to DNS queries in the "consul." domain. This flag can be used to change that domain. All queries in this domain are assumed to be handled by Consul and will not be recursively resolved.

• -alt-domain - This flag allows Consul to respond to DNS queries in an alternate domain, in addition to the primary domain. If unset, no alternate domain is used.

• -enable-script-checks This controls whether health checks that execute scripts are enabled on this agent, and defaults to false so operators must opt-in to allowing these. This was added in Consul 0.9.0.

  ~> Security Warning: Enabling script checks in some configurations may introduce a remote execution vulnerability which is known to be targeted by malware. We strongly recommend -enable-local-script-checks instead. See this blog post for more details.

• -enable-local-script-checks Like enable_script_checks, but only enable them when they are defined in the local configuration files. Script checks defined in HTTP API registrations will still not be allowed.

• -encrypt - Specifies the secret key to use for encryption of Consul network traffic. This key must be 32-bytes that are Base64-encoded. The easiest way to create an encryption key is to use consul keygen. All nodes within a cluster must share the same encryption key to communicate. The provided key is automatically persisted to the data directory and loaded automatically whenever the agent is restarted. This means that to encrypt Consul's gossip protocol, this option only needs to be provided once on each agent's initial startup sequence. If it is provided after Consul has been initialized with an encryption key, then the provided key is ignored and a warning will be displayed.

• -grpc-port - the gRPC API port to listen on. Default -1 (gRPC disabled). See ports documentation for more detail.

• -hcl - A HCL configuration fragment. This HCL configuration fragment is appended to the configuration and allows to specify the full range of options of a config file on the command line. This option can be specified multiple times. This was added in Consul 1.0.

• -http-port - the HTTP API port to listen on. This overrides the default port 8500. This option is very useful when deploying Consul to an environment which communicates the HTTP port through the environment e.g. PaaS like CloudFoundry, allowing you to set the port directly via a Procfile.

• -log-file - to redirect all the Consul agent log messages to a file. This can be specified with the complete path along with the name of the log. In case the path doesn't have the filename, the filename defaults to consul-{timestamp}.log. Can be combined with -log-rotate-bytes and -log-rotate-duration for a fine-grained log rotation experience.

• -log-rotate-bytes - to specify the number of bytes that should be written to a log before it needs to be rotated. Unless specified, there is no limit to the number of bytes that can be written to a log file.

• -log-rotate-duration - to specify the maximum duration a log should be written to before it needs to be rotated. Must be a duration value such as 30s. Defaults to 24h.

• -log-rotate-max-files - to specify the maximum number of older log file archives to keep. Defaults to 0 (no files are ever deleted). Set to -1 to disable rotation and discard all log files.

• -join - Address of another agent to join upon starting up. This can be specified multiple times to specify multiple agents to join. If Consul is unable to join with any of the specified addresses, agent startup will fail. By default, the agent won't join any nodes when it starts up. Note that using retry_join could be more appropriate to help mitigate node startup race conditions when automating a Consul cluster deployment.

  In Consul 1.1.0 and later this can be set to a go-sockaddr template

• -retry-join - Similar to -join but allows retrying a join if the first attempt fails. This is useful for cases where you know the address will eventually be available. The list can contain IPv4, IPv6, or DNS addresses. In Consul 1.1.0 and later this can be set to a go-sockaddr template. If Consul is running on the non-default Serf LAN port, this must be specified as well. IPv6 must use the "bracketed" syntax. If multiple values are given, they are tried and retried in the order listed until the first succeeds. Here are some examples:

  # Using a DNS entry
  $ consul agent -retry-join "consul.domain.internal"
  

  # Using IPv4
  $ consul agent -retry-join "10.0.4.67"
  

  # Using IPv6
  $ consul agent -retry-join "[::1]:8301"
  

  Cloud Auto-Joining

  As of Consul 0.9.1, retry-join accepts a unified interface using the go-discover library for doing automatic cluster joining using cloud metadata. For more information, see the Cloud Auto-join page.

  # Using Cloud Auto-Joining
  $ consul agent -retry-join "provider=aws tag_key=..."
  

• -retry-interval - Time to wait between join attempts. Defaults to 30s.

• -retry-max - The maximum number of -join attempts to be made before exiting with return code 1. By default, this is set to 0 which is interpreted as infinite retries.

• -join-wan - Address of another wan agent to join upon starting up. This can be specified multiple times to specify multiple WAN agents to join. If Consul is unable to join with any of the specified addresses, agent startup will fail. By default, the agent won't -join-wan any nodes when it starts up.

  In Consul 1.1.0 and later this can be set to a go-sockaddr template.

• -retry-join-wan - Similar to retry-join but allows retrying a wan join if the first attempt fails. This is useful for cases where we know the address will become available eventually. As of Consul 0.9.3 Cloud Auto-Joining is supported as well.

  In Consul 1.1.0 and later this can be set to a go-sockaddr template

• -retry-interval-wan - Time to wait between -join-wan attempts. Defaults to 30s.

• -retry-max-wan - The maximum number of -join-wan attempts to be made before exiting with return code 1. By default, this is set to 0 which is interpreted as infinite retries.

• -log-level - The level of logging to show after the Consul agent has started. This defaults to "info". The available log levels are "trace", "debug", "info", "warn", and "err". You can always connect to an agent via consul monitor and use any log level. Also, the log level can be changed during a config reload.

• -node - The name of this node in the cluster. This must be unique within the cluster. By default this is the hostname of the machine.

• -node-id - Available in Consul 0.7.3 and later, this is a unique identifier for this node across all time, even if the name of the node or address changes. This must be in the form of a hex string, 36 characters long, such as adf4238a-882b-9ddc-4a9d-5b6758e4159e. If this isn't supplied, which is the most common case, then the agent will generate an identifier at startup and persist it in the data directory so that it will remain the same across agent restarts. Information from the host will be used to generate a deterministic node ID if possible, unless -disable-host-node-id is set to true.

• -node-meta - Available in Consul 0.7.3 and later, this specifies an arbitrary metadata key/value pair to associate with the node, of the form key:value. This can be specified multiple times. Node metadata pairs have the following restrictions:

  • A maximum of 64 key/value pairs can be registered per node.
  • Metadata keys must be between 1 and 128 characters (inclusive) in length
  • Metadata keys must contain only alphanumeric, -, and _ characters.
  • Metadata keys must not begin with the consul- prefix; that is reserved for internal use by Consul.
  • Metadata values must be between 0 and 512 (inclusive) characters in length.
  • Metadata values for keys beginning with rfc1035- are encoded verbatim in DNS TXT requests, otherwise the metadata kv-pair is encoded according RFC1464.

• -pid-file - This flag provides the file path for the agent to store its PID. This is useful for sending signals (for example, SIGINT to close the agent or SIGHUP to update check definite

• -protocol - The Consul protocol version to use. Consul agents speak protocol 2 by default, however agents will automatically use protocol >2 when speaking to compatible agents. This should be set only when upgrading. You can view the protocol versions supported by Consul by running consul -v.

• -raft-protocol - This controls the internal version of the Raft consensus protocol used for server communications. This must be set to 3 in order to gain access to Autopilot features, with the exception of cleanup_dead_servers. Defaults to 3 in Consul 1.0.0 and later (defaulted to 2 previously). See Raft Protocol Version Compatibility for more details.

• -recursor - Specifies the address of an upstream DNS server. This option may be provided multiple times, and is functionally equivalent to the recursors configuration option.

• -rejoin - When provided, Consul will ignore a previous leave and attempt to rejoin the cluster when starting. By default, Consul treats leave as a permanent intent and does not attempt to join the cluster again when starting. This flag allows the previous state to be used to rejoin the cluster.

• -segment - (Enterprise-only) This flag is used to set the name of the network segment the agent belongs to. An agent can only join and communicate with other agents within its network segment. See the Network Segments Guide for more details. By default, this is an empty string, which is the default network segment.

• -serf-lan-port - the Serf LAN port to listen on. This overrides the default Serf LAN port 8301. This is available in Consul 1.2.2 and later.

• -serf-wan-port - the Serf WAN port to listen on. This overrides the default Serf WAN port 8302. This is available in Consul 1.2.2 and later.

• -server - This flag is used to control if an agent is in server or client mode. When provided, an agent will act as a Consul server. Each Consul cluster must have at least one server and ideally no more than 5 per datacenter. All servers participate in the Raft consensus algorithm to ensure that transactions occur in a consistent, linearizable manner. Transactions modify cluster state, which is maintained on all server nodes to ensure availability in the case of node failure. Server nodes also participate in a WAN gossip pool with server nodes in other datacenters. Servers act as gateways to other datacenters and forward traffic as appropriate.

• -server-port - the server RPC port to listen on. This overrides the default server RPC port 8300. This is available in Consul 1.2.2 and later.

• -non-voting-server - (Enterprise-only) This flag is used to make the server not participate in the Raft quorum, and have it only receive the data replication stream. This can be used to add read scalability to a cluster in cases where a high volume of reads to servers are needed.

• -syslog - This flag enables logging to syslog. This is only supported on Linux and OSX. It will result in an error if provided on Windows.

• -ui - Enables the built-in web UI server and the required HTTP routes. This eliminates the need to maintain the Consul web UI files separately from the binary.

• -ui-dir - This flag provides the directory containing the Web UI resources for Consul. This will automatically enable the Web UI. The directory must be readable to the agent. Starting with Consul version 0.7.0 and later, the Web UI assets are included in the binary so this flag is no longer necessary; specifying only the -ui flag is enough to enable the Web UI. Specifying both the '-ui' and '-ui-dir' flags will result in an error.

• -ui-content-path - This flag provides the option to change the path the Consul UI loads from and will be displayed in the browser. By default, the path is /ui/, for example http://localhost:8500/ui/. Only alphanumerics, -, and _ are allowed in a custom path. /v1/ is not allowed as it would overwrite the API endpoint.

Configuration Files

In addition to the command-line options, configuration can be put into files. This may be easier in certain situations, for example when Consul is being configured using a configuration management system.

The configuration files are JSON formatted, making them easily readable and editable by both humans and computers. The configuration is formatted as a single JSON object with configuration within it.

Configuration files are used for more than just setting up the agent, they are also used to provide check and service definitions. These are used to announce the availability of system servers to the rest of the cluster. They are documented separately under check configuration and service configuration respectively. The service and check definitions support being updated during a reload.

Example Configuration File

{
  "datacenter": "east-aws",
  "data_dir": "/opt/consul",
  "log_level": "INFO",
  "node_name": "foobar",
  "server": true,
  "watches": [
    {
        "type": "checks",
        "handler": "/usr/bin/health-check-handler.sh"
    }
  ],
  "telemetry": {
     "statsite_address": "127.0.0.1:2180"
  }
}

Example Configuration File, with TLS

{
  "datacenter": "east-aws",
  "data_dir": "/opt/consul",
  "log_level": "INFO",
  "node_name": "foobar",
  "server": true,
  "addresses": {
    "https": "0.0.0.0"
  },
  "ports": {
    "https": 8501
  },
  "key_file": "/etc/pki/tls/private/my.key",
  "cert_file": "/etc/pki/tls/certs/my.crt",
  "ca_file": "/etc/pki/tls/certs/ca-bundle.crt"
}

See, especially, the use of the ports setting:

"ports": {
  "https": 8501
}

Consul will not enable TLS for the HTTP API unless the https port has been assigned a port number > 0. We recommend using 8501 for https as this default will automatically work with some tooling.

Configuration Key Reference

• acl - This object allows a number of sub-keys to be set which controls the ACL system. Configuring the ACL system within the ACL stanza was added in Consul 1.4.0

  The following sub-keys are available:

  • enabled - Enables ACLs.

  • policy_ttl - Used to control Time-To-Live caching of ACL policies. By default, this is 30 seconds. This setting has a major performance impact: reducing it will cause more frequent refreshes while increasing it reduces the number of refreshes. However, because the caches are not actively invalidated, ACL policy may be stale up to the TTL value.

  • role_ttl - Used to control Time-To-Live caching of ACL roles. By default, this is 30 seconds. This setting has a major performance impact: reducing it will cause more frequent refreshes while increasing it reduces the number of refreshes. However, because the caches are not actively invalidated, ACL role may be stale up to the TTL value.

  • token_ttl - Used to control Time-To-Live caching of ACL tokens. By default, this is 30 seconds. This setting has a major performance impact: reducing it will cause more frequent refreshes while increasing it reduces the number of refreshes. However, because the caches are not actively invalidated, ACL token may be stale up to the TTL value.

  • down_policy - Either "allow", "deny", "extend-cache" or "async-cache"; "extend-cache" is the default. In the case that a policy or token cannot be read from the primary_datacenter or leader node, the down policy is applied. In "allow" mode, all actions are permitted, "deny" restricts all operations, and "extend-cache" allows any cached objects to be used, ignoring their TTL values. If a non-cached ACL is used, "extend-cache" acts like "deny". The value "async-cache" acts the same way as "extend-cache" but performs updates asynchronously when ACL is present but its TTL is expired, thus, if latency is bad between the primary and secondary datacenters, latency of operations is not impacted.

  • default_policy - Either "allow" or "deny"; defaults to "allow" but this will be changed in a future major release. The default policy controls the behavior of a token when there is no matching rule. In "allow" mode, ACLs are a blacklist: any operation not specifically prohibited is allowed. In "deny" mode, ACLs are a whitelist: any operation not specifically allowed is blocked. Note: this will not take effect until you've enabled ACLs.

  • enable_key_list_policy - Either "enabled" or "disabled", defaults to "disabled". When enabled, the list permission will be required on the prefix being recursively read from the KV store. Regardless of being enabled, the full set of KV entries under the prefix will be filtered to remove any entries that the request's ACL token does not grant at least read permissions. This option is only available in Consul 1.0 and newer.

  • enable_token_replication - By default secondary Consul datacenters will perform replication of only ACL policies and roles. Setting this configuration will will enable ACL token replication and allow for the creation of both local tokens and auth methods in connected secondary datacenters.

  • enable_token_persistence - Either true or false. When true tokens set using the API will be persisted to disk and reloaded when an agent restarts.

  • tokens - This object holds all of the configured ACL tokens for the agents usage.

    • master - Only used for servers in the primary_datacenter. This token will be created with management-level permissions if it does not exist. It allows operators to bootstrap the ACL system with a token Secret ID that is well-known.
      
      The master token is only installed when a server acquires cluster leadership. If you would like to install or change the acl_master_token, set the new value for master in the configuration for all servers. Once this is done, restart the current leader to force a leader election. If the master token is not supplied, then the servers do not create a master token. When you provide a value, it should be a UUID. To maintain backwards compatibility and an upgrade path this restriction is not currently enforced but will be in a future major Consul release.

    • default - When provided, the agent will use this token when making requests to the Consul servers. Clients can override this token on a per-request basis by providing the "?token" query parameter. When not provided, the empty token, which maps to the 'anonymous' ACL token, is used.

    • agent - Used for clients and servers to perform internal operations. If this isn't specified, then the default will be used.
      
      This token must at least have write access to the node name it will register as in order to set any of the node-level information in the catalog such as metadata, or the node's tagged addresses.

    • agent_master - Used to access agent endpoints that require agent read or write privileges, or node read privileges, even if Consul servers aren't present to validate any tokens. This should only be used by operators during outages, regular ACL tokens should normally be used by applications.

    • replication - The ACL token used to authorize secondary datacenters with the primary datacenter for replication operations. This token is required for servers outside the primary_datacenter when ACLs are enabled. This token may be provided later using the agent token API on each server. This token must have at least "read" permissions on ACL data but if ACL token replication is enabled then it must have "write" permissions. This also enables Connect replication, for which the token will require both operator "write" and intention "read" permissions for replicating CA and Intention data.

• acl_datacenter - This field is deprecated in Consul 1.4.0. See the primary_datacenter field instead.

  This designates the datacenter which is authoritative for ACL information. It must be provided to enable ACLs. All servers and datacenters must agree on the ACL datacenter. Setting it on the servers is all you need for cluster-level enforcement, but for the APIs to forward properly from the clients, it must be set on them too. In Consul 0.8 and later, this also enables agent-level enforcement of ACLs. Please see the ACL Guide for more details.

• acl_default_policy - Deprecated in Consul 1.4.0. See the acl.default_policy field instead. Either "allow" or "deny"; defaults to "allow". The default policy controls the behavior of a token when there is no matching rule. In "allow" mode, ACLs are a blacklist: any operation not specifically prohibited is allowed. In "deny" mode, ACLs are a whitelist: any operation not specifically allowed is blocked. Note: this will not take effect until you've set primary_datacenter to enable ACL support.

• acl_down_policy - **Deprecated in Consul 1.4.0. See the acl.down_policy field instead.**Either "allow", "deny", "extend-cache" or "async-cache"; "extend-cache" is the default. In the case that the policy for a token cannot be read from the primary_datacenter or leader node, the down policy is applied. In "allow" mode, all actions are permitted, "deny" restricts all operations, and "extend-cache" allows any cached ACLs to be used, ignoring their TTL values. If a non-cached ACL is used, "extend-cache" acts like "deny". The value "async-cache" acts the same way as "extend-cache" but performs updates asynchronously when ACL is present but its TTL is expired, thus, if latency is bad between ACL authoritative and other datacenters, latency of operations is not impacted.

• acl_agent_master_token - Deprecated in Consul 1.4.0. See the acl.tokens.agent_master field instead. Used to access agent endpoints that require agent read or write privileges, or node read privileges, even if Consul servers aren't present to validate any tokens. This should only be used by operators during outages, regular ACL tokens should normally be used by applications. This was added in Consul 0.7.2 and is only used when acl_enforce_version_8 is set to true.

• acl_agent_token - Deprecated in Consul 1.4.0. See the acl.tokens.agent field instead. Used for clients and servers to perform internal operations. If this isn't specified, then the acl_token will be used. This was added in Consul 0.7.2.

  This token must at least have write access to the node name it will register as in order to set any of the node-level information in the catalog such as metadata, or the node's tagged addresses.

• acl_enforce_version_8 - Deprecated in Consul 1.4.0 Used for clients and servers to determine if enforcement should occur for new ACL policies being previewed before Consul 0.8. Added in Consul 0.7.2, this defaults to false in versions of Consul prior to 0.8, and defaults to true in Consul 0.8 and later. This helps ease the transition to the new ACL features by allowing policies to be in place before enforcement begins.

• acl_master_token - Deprecated in Consul 1.4.0. See the acl.tokens.master field instead. Only used for servers in the primary_datacenter. This token will be created with management-level permissions if it does not exist. It allows operators to bootstrap the ACL system with a token ID that is well-known.

  The acl_master_token is only installed when a server acquires cluster leadership. If you would like to install or change the acl_master_token, set the new value for acl_master_token in the configuration for all servers. Once this is done, restart the current leader to force a leader election. If the acl_master_token is not supplied, then the servers do not create a master token. When you provide a value, it can be any string value. Using a UUID would ensure that it looks the same as the other tokens, but isn't strictly necessary.

• acl_replication_token - Deprecated in Consul 1.4.0. See the acl.tokens.replication field instead. Only used for servers outside the primary_datacenter running Consul 0.7 or later. When provided, this will enable ACL replication using this ACL replication using this token to retrieve and replicate the ACLs to the non-authoritative local datacenter. In Consul 0.9.1 and later you can enable ACL replication using enable_acl_replication and then set the token later using the agent token API on each server. If the acl_replication_token is set in the config, it will automatically set enable_acl_replication to true for backward compatibility.

  If there's a partition or other outage affecting the authoritative datacenter, and the acl_down_policy is set to "extend-cache", tokens not in the cache can be resolved during the outage using the replicated set of ACLs.

• acl_token - Deprecated in Consul 1.4.0. See the acl.tokens.default field instead. When provided, the agent will use this token when making requests to the Consul servers. Clients can override this token on a per-request basis by providing the "?token" query parameter. When not provided, the empty token, which maps to the 'anonymous' ACL policy, is used.

• acl_ttl - **Deprecated in Consul 1.4.0. See the acl.token_ttl field instead.**Used to control Time-To-Live caching of ACLs. By default, this is 30 seconds. This setting has a major performance impact: reducing it will cause more frequent refreshes while increasing it reduces the number of refreshes. However, because the caches are not actively invalidated, ACL policy may be stale up to the TTL value.

• addresses - This is a nested object that allows setting bind addresses. In Consul 1.0 and later these can be set to a space-separated list of addresses to bind to, or a go-sockaddr template that can potentially resolve to multiple addresses.

  http, https and grpc all support binding to a Unix domain socket. A socket can be specified in the form unix:///path/to/socket. A new domain socket will be created at the given path. If the specified file path already exists, Consul will attempt to clear the file and create the domain socket in its place. The permissions of the socket file are tunable via the unix_sockets config construct.

  When running Consul agent commands against Unix socket interfaces, use the -http-addr argument to specify the path to the socket. You can also place the desired values in the CONSUL_HTTP_ADDR environment variable.

  For TCP addresses, the environment variable value should be an IP address with the port. For example: 10.0.0.1:8500 and not 10.0.0.1. However, ports are set separately in the ports structure when defining them in a configuration file.

  The following keys are valid:

  • dns - The DNS server. Defaults to client_addr
  • http - The HTTP API. Defaults to client_addr
  • https - The HTTPS API. Defaults to client_addr
  • grpc - The gRPC API. Defaults to client_addr

• advertise_addr Equivalent to the -advertise command-line flag.

• serf_wan Equivalent to the -serf-wan-bind command-line flag.

• serf_lan Equivalent to the -serf-lan-bind command-line flag.

• advertise_addr_wan Equivalent to the -advertise-wan command-line flag.

• autopilot Added in Consul 0.8, this object allows a number of sub-keys to be set which can configure operator-friendly settings for Consul servers. When these keys are provided as configuration, they will only be respected on bootstrapping. If they are not provided, the defaults will be used. In order to change the value of these options after bootstrapping, you will need to use the Consul Operator Autopilot command. For more information about Autopilot, see the Autopilot Guide.

  The following sub-keys are available:

  • cleanup_dead_servers - This controls the automatic removal of dead server nodes periodically and whenever a new server is added to the cluster. Defaults to true.

  • last_contact_threshold - Controls the maximum amount of time a server can go without contact from the leader before being considered unhealthy. Must be a duration value such as 10s. Defaults to 200ms.

  • max_trailing_logs - Controls the maximum number of log entries that a server can trail the leader by before being considered unhealthy. Defaults to 250.

  • min_quorum - Sets the minimum number of servers necessary in a cluster before autopilot can prune dead servers. There is no default.

  • server_stabilization_time - Controls the minimum amount of time a server must be stable in the 'healthy' state before being added to the cluster. Only takes effect if all servers are running Raft protocol version 3 or higher. Must be a duration value such as 30s. Defaults to 10s.

  • redundancy_zone_tag - (Enterprise-only) This controls the -node-meta key to use when Autopilot is separating servers into zones for redundancy. Only one server in each zone can be a voting member at one time. If left blank (the default), this feature will be disabled.

  • disable_upgrade_migration - (Enterprise-only) If set to true, this setting will disable Autopilot's upgrade migration strategy in Consul Enterprise of waiting until enough newer-versioned servers have been added to the cluster before promoting any of them to voters. Defaults to false.

  • upgrade_version_tag - (Enterprise-only) The node_meta tag to use for version info when performing upgrade migrations. If this is not set, the Consul version will be used.

• auto_encrypt This object allows setting options for the auto_encrypt feature.

  The following sub-keys are available:

  • allow_tls (Defaults to false) This option enables auto_encrypt on the servers and allows them to automatically distribute certificates from the Connect CA to the clients. If enabled, the server can accept incoming connections from both the built-in CA and the Connect CA, as well as their certificates. Note, the server will only present the built-in CA and certificate, which the client can verify using the CA it received from auto_encrypt endpoint. If disabled, a client configured with auto_encrypt.tls will be unable to start.

  • tls (Defaults to false) Allows the client to request the Connect CA and certificates from the servers, for encrypting RPC communication. The client will make the request to any servers listed in the -join or -retry-join option. This requires that every server to have auto_encrypt.allow_tls enabled. When both auto_encrypt options are used, it allows clients to receive certificates that are generated on the servers. If the -server-port is not the default one, it has to be provided to the client as well. Usually this is discovered through LAN gossip, but auto_encrypt provision happens before the information can be distributed through gossip. The most secure auto_encrypt setup is when the client is provided with the built-in CA, verify_server_hostname is turned on, and when an ACL token with node.write permissions is setup. It is also possible to use auto_encrypt with a CA and ACL, but without verify_server_hostname, or only with a ACL enabled, or only with CA and verify_server_hostname, or only with a CA, or finally without a CA and without ACL enabled. In any case, the communication to the auto_encrypt endpoint is always TLS encrypted.

• bootstrap Equivalent to the -bootstrap command-line flag.

• bootstrap_expect Equivalent to the -bootstrap-expect command-line flag.

• bind_addr Equivalent to the -bind command-line flag.

• ca_file This provides a file path to a PEM-encoded certificate authority. The certificate authority is used to check the authenticity of client and server connections with the appropriate verify_incoming or verify_outgoing flags.

• ca_path This provides a path to a directory of PEM-encoded certificate authority files. These certificate authorities are used to check the authenticity of client and server connections with the appropriate verify_incoming or verify_outgoing flags.

• cert_file This provides a file path to a PEM-encoded certificate. The certificate is provided to clients or servers to verify the agent's authenticity. It must be provided along with key_file.

• check_update_interval This interval controls how often check output from checks in a steady state is synchronized with the server. By default, this is set to 5 minutes ("5m"). Many checks which are in a steady state produce slightly different output per run (timestamps, etc) which cause constant writes. This configuration allows deferring the sync of check output for a given interval to reduce write pressure. If a check ever changes state, the new state and associated output is synchronized immediately. To disable this behavior, set the value to "0s".

• client_addr Equivalent to the -client command-line flag.

• config_entries This object allows setting options for centralized config entries.

  The following sub-keys are available:

  • bootstrap This is a list of inlined config entries to insert into the state store when the Consul server gains leadership. This option is only applicable to server nodes. Each bootstrap entry will be created only if it does not exist. When reloading, any new entries that have been added to the configuration will be processed. See the configuration entry docs for more details about the contents of each entry.

• connect This object allows setting options for the Connect feature.

  The following sub-keys are available:

  • enabled Controls whether Connect features are enabled on this agent. Should be enabled on all clients and servers in the cluster in order for Connect to function properly. Defaults to false.

  • ca_provider Controls which CA provider to use for Connect's CA. Currently only the consul and vault providers are supported. This is only used when initially bootstrapping the cluster. For an existing cluster, use the Update CA Configuration Endpoint.

  • ca_config An object which allows setting different config options based on the CA provider chosen. This is only used when initially bootstrapping the cluster. For an existing cluster, use the Update CA Configuration Endpoint.

    The following providers are supported:

    Consul CA Provider (ca_provider = "consul")

    • private_key The PEM contents of the private key to use for the CA.

    • root_cert The PEM contents of the root certificate to use for the CA.

    Vault CA Provider (ca_provider = "vault")

    • address The address of the Vault server to connect to.

    • token The Vault token to use.

    • root_pki_path The path to use for the root CA pki backend in Vault. This can be an existing backend with a CA already configured, or a blank/unmounted backend in which case Connect will automatically mount/generate the CA. The Vault token given above must have sudo access to this backend, as well as permission to mount the backend at this path if it is not already mounted.

    • intermediate_pki_path The path to use for the temporary intermediate CA pki backend in Vault. Connect will overwrite any data at this path in order to generate a temporary intermediate CA. The Vault token given above must have write access to this backend, as well as permission to mount the backend at this path if it is not already mounted.

    Common CA Config Options

    There are also a number of common configuration options supported by all providers:

    • csr_max_concurrent Sets a limit on how many Certificate Signing Requests will be processed concurrently. Defaults to 0 (disabled). This is useful when you have more than one or two cores available to the server. For example on an 8 core server, setting this to 1 will ensure that even during a CA rotation no more than one server core on the leader will be consumed at a time with generating new certificates. Setting this is recommended instead of csr_max_per_second where you know there are multiple cores available since it is simpler to reason about limiting CSR resources this way without artificially slowing down rotations. Added in 1.4.1.

    • csr_max_per_second Sets a rate limit on the maximum number of Certificate Signing Requests (CSRs) the servers will accept. This is used to prevent CA rotation from causing unbounded CPU usage on servers. It defaults to 50 which is conservative - a 2017 Macbook can process about 100 per second using only ~40% of one CPU core - but sufficient for deployments up to ~1500 service instances before the time it takes to rotate is impacted. For larger deployments we recommend increasing this based on the expected number of server instances and server resources, or use csr_max_concurrent instead if servers have more than one core. Setting this to zero disables rate limiting. Added in 1.4.1.

    • leaf_cert_ttl The upper bound on the lease duration of a leaf certificate issued for a service. In most cases a new leaf certificate will be requested by a proxy before this limit is reached. This is also the effective limit on how long a server outage can last (with no leader) before network connections will start being rejected, and as a result the defaults is 72h to last through a weekend without intervention. This value cannot be lower than 1 hour or higher than 1 year.

      This value is also used when rotating out old root certificates from the cluster. When a root certificate has been inactive (rotated out) for more than twice the current leaf_cert_ttl, it will be removed from the trusted list.

    • private_key_type The type of key to generate for this CA. This is only used when the provider is generating a new key. If private_key is set for the Consul provider, or existing root or intermediate PKI paths given for Vault then this will be ignored. Currently supported options are ec or rsa. Default is ec.

      It is required that all servers in a Datacenter have the same config for the CA. It is recommended that servers in different Datacenters have the same CA config for key type and size although the built-in CA and Vault provider will both allow mixed CA key types.

      Some CA providers (currently Vault) will not allow cross-signing a new CA certificate with a different key type. This means that if you migrate from an RSA-keyed Vault CA to an EC-keyed CA from any provider, you may have to proceed without cross-signing which risks temporary connection issues for workloads during the new certificate rollout. We highly recommend testing this outside of production to understand the impact and suggest sticking to same key type where possible.

      Note that this only affects CA keys generated by the provider. Leaf certificate keys are always EC 256 regardless of the CA configuration.

    • private_key_bits The length of key to generate for this CA. This is only used when the provider is generating a new key. If private_key is set for the Consul provider, or existing root or intermediate PKI paths given for Vault then this will be ignored.

      Currently supported values are:

      • private_key_type = ec (default): 224, 256, 384, 521 corresponding to the NIST P-* curves of the same name.
      • private_key_type = rsa: 2048, 4096

• datacenter Equivalent to the -datacenter command-line flag.

• data_dir Equivalent to the -data-dir command-line flag.

• disable_anonymous_signature Disables providing an anonymous signature for de-duplication with the update check. See disable_update_check.

• disable_host_node_id Equivalent to the -disable-host-node-id command-line flag.

• disable_http_unprintable_char_filter Defaults to false. Consul 1.0.3 fixed a potential security vulnerability where malicious users could craft KV keys with unprintable chars that would confuse operators using the CLI or UI into taking wrong actions. Users who had data written in older versions of Consul that did not have this restriction will be unable to delete those values by default in 1.0.3 or later. This setting enables those users to temporarily disable the filter such that delete operations can work on those keys again to get back to a healthy state. It is strongly recommended that this filter is not disabled permanently as it exposes the original security vulnerability.

• disable_remote_exec Disables support for remote execution. When set to true, the agent will ignore any incoming remote exec requests. In versions of Consul prior to 0.8, this defaulted to false. In Consul 0.8 the default was changed to true, to make remote exec opt-in instead of opt-out.

• disable_update_check Disables automatic checking for security bulletins and new version releases. This is disabled in Consul Enterprise.

• discard_check_output Discards the output of health checks before storing them. This reduces the number of writes to the Consul raft log in environments where health checks have volatile output like timestamps, process ids, ...

• discovery_max_stale - Enables stale requests for all service discovery HTTP endpoints. This is equivalent to the max_stale configuration for DNS requests. If this value is zero (default), all service discovery HTTP endpoints are forwarded to the leader. If this value is greater than zero, any Consul server can handle the service discovery request. If a Consul server is behind the leader by more than discovery_max_stale, the query will be re-evaluated on the leader to get more up-to-date results. Consul agents also add a new X-Consul-Effective-Consistency response header which indicates if the agent did a stale read. discover-max-stale was introduced in Consul 1.0.7 as a way for Consul operators to force stale requests from clients at the agent level, and defaults to zero which matches default consistency behavior in earlier Consul versions.

• dns_config This object allows a number of sub-keys to be set which can tune how DNS queries are serviced. See this guide on DNS caching for more detail.

  The following sub-keys are available:

  • allow_stale - Enables a stale query for DNS information. This allows any Consul server, rather than only the leader, to service the request. The advantage of this is you get linear read scalability with Consul servers. In versions of Consul prior to 0.7, this defaulted to false, meaning all requests are serviced by the leader, providing stronger consistency but less throughput and higher latency. In Consul 0.7 and later, this defaults to true for better utilization of available servers.

  • max_stale - When allow_stale is specified, this is used to limit how stale results are allowed to be. If a Consul server is behind the leader by more than max_stale, the query will be re-evaluated on the leader to get more up-to-date results. Prior to Consul 0.7.1 this defaulted to 5 seconds; in Consul 0.7.1 and later this defaults to 10 years ("87600h") which effectively allows DNS queries to be answered by any server, no matter how stale. In practice, servers are usually only milliseconds behind the leader, so this lets Consul continue serving requests in long outage scenarios where no leader can be elected.

  • node_ttl - By default, this is "0s", so all node lookups are served with a 0 TTL value. DNS caching for node lookups can be enabled by setting this value. This should be specified with the "s" suffix for second or "m" for minute.

  • service_ttl - This is a sub-object which allows for setting a TTL on service lookups with a per-service policy. The "*" wildcard service can be used when there is no specific policy available for a service. By default, all services are served with a 0 TTL value. DNS caching for service lookups can be enabled by setting this value.

  • enable_truncate - If set to true, a UDP DNS query that would return more than 3 records, or more than would fit into a valid UDP response, will set the truncated flag, indicating to clients that they should re-query using TCP to get the full set of records.

  • only_passing - If set to true, any nodes whose health checks are warning or critical will be excluded from DNS results. If false, the default, only nodes whose healthchecks are failing as critical will be excluded. For service lookups, the health checks of the node itself, as well as the service-specific checks are considered. For example, if a node has a health check that is critical then all services on that node will be excluded because they are also considered critical.

  • recursor_timeout - Timeout used by Consul when recursively querying an upstream DNS server. See recursors for more details. Default is 2s. This is available in Consul 0.7 and later.

  • disable_compression - If set to true, DNS responses will not be compressed. Compression was added and enabled by default in Consul 0.7.

  • udp_answer_limit - Limit the number of resource records contained in the answer section of a UDP-based DNS response. This parameter applies only to UDP DNS queries that are less than 512 bytes. This setting is deprecated and replaced in Consul 1.0.7 by a_record_limit.

  • a_record_limit - Limit the number of resource records contained in the answer section of a A, AAAA or ANY DNS response (both TCP and UDP). When answering a question, Consul will use the complete list of matching hosts, shuffle the list randomly, and then limit the number of answers to a_record_limit (default: no limit). This limit does not apply to SRV records.

    In environments where RFC 3484 Section 6 Rule 9 is implemented and enforced (i.e. DNS answers are always sorted and therefore never random), clients may need to set this value to 1 to preserve the expected randomized distribution behavior (note: RFC 3484 has been obsoleted by RFC 6724 and as a result it should be increasingly uncommon to need to change this value with modern resolvers).

  • enable_additional_node_meta_txt - When set to true, Consul will add TXT records for Node metadata into the Additional section of the DNS responses for several query types such as SRV queries. When set to false those records are not emitted. This does not impact the behavior of those same TXT records when they would be added to the Answer section of the response like when querying with type TXT or ANY. This defaults to true.

  • soa Allow to tune the setting set up in SOA. Non specified values fallback to their default values, all values are integers and expressed as seconds.
    
    The following settings are available:

    • expire - Configure SOA Expire duration in seconds, default value is 86400, ie: 24 hours.

    • min_ttl - Configure SOA DNS minimum TTL. As explained in RFC-2308 this also controls negative cache TTL in most implementations. Default value is 0, ie: no minimum delay or negative TTL.

    • refresh - Configure SOA Refresh duration in seconds, default value is 3600, ie: 1 hour.

    • retry - Configures the Retry duration expressed in seconds, default value is 600, ie: 10 minutes.

  • use_cache - When set to true, DNS resolution will use the agent cache described in agent caching. This setting affects all service and prepared queries DNS requests. Implies allow_stale

  • cache_max_age - When use_cache is enabled, the agent will attempt to re-fetch the result from the servers if the cached value is older than this duration. See: agent caching.

  • prefer_namespace - (Enterprise Only) When set to true, in a DNS query for a service, the label between the domain and the service label will be treated as a namespace name instead of a datacenter. When set to false, the default, the behavior will be the same as non-Enterprise versions and will assume the label is the datacenter. See: this section for more details.

• domain Equivalent to the -domain command-line flag.

• enable_acl_replication When set on a Consul server, enables ACL replication without having to set the replication token via acl_replication_token. Instead, enable ACL replication and then introduce the token using the agent token API on each server. See acl_replication_token for more details.

• enable_agent_tls_for_checks When set, uses a subset of the agent's TLS configuration (key_file, cert_file, ca_file, ca_path, and server_name) to set up the client for HTTP or gRPC health checks. This allows services requiring 2-way TLS to be checked using the agent's credentials. This was added in Consul 1.0.1 and defaults to false.

• enable_central_service_config When set, the Consul agent will look for any centralized service configurations that match a registering service instance. If it finds any, the agent will merge the centralized defaults with the service instance configuration. This allows for things like service protocol or proxy configuration to be defined centrally and inherited by any affected service registrations.

• enable_debug When set, enables some additional debugging features. Currently, this is only used to access runtime profiling HTTP endpoints, which are available with an operator:read ACL regardless of the value of enable_debug.

• enable_script_checks Equivalent to the -enable-script-checks command-line flag.

  ~> Security Warning: Enabling script checks in some configurations may introduce a remote execution vulnerability which is known to be targeted by malware. We strongly recommend enable_local_script_checks instead. See this blog post for more details.

• enable_local_script_checks Equivalent to the -enable-local-script-checks command-line flag.

• enable_syslog Equivalent to the -syslog command-line flag.

• encrypt Equivalent to the -encrypt command-line flag.

• encrypt_verify_incoming - This is an optional parameter that can be used to disable enforcing encryption for incoming gossip in order to upshift from unencrypted to encrypted gossip on a running cluster. See [this section] (/docs/agent/encryption.html#configuring-gossip-encryption-on-an-existing-cluster) for more information. Defaults to true.

• encrypt_verify_outgoing - This is an optional parameter that can be used to disable enforcing encryption for outgoing gossip in order to upshift from unencrypted to encrypted gossip on a running cluster. See [this section] (/docs/agent/encryption.html#configuring-gossip-encryption-on-an-existing-cluster) for more information. Defaults to true.

• disable_keyring_file - Equivalent to the -disable-keyring-file command-line flag.

• gossip_lan - (Advanced) This object contains a number of sub-keys which can be set to tune the LAN gossip communications. These are only provided for users running especially large clusters that need fine tuning and are prepared to spend significant effort correctly tuning them for their environment and workload. Tuning these improperly can cause Consul to fail in unexpected ways. The default values are appropriate in almost all deployments.

  • gossip_nodes - The number of random nodes to send gossip messages to per gossip_interval. Increasing this number causes the gossip messages to propagate across the cluster more quickly at the expense of increased bandwidth. The default is 3.

  • gossip_interval - The interval between sending messages that need to be gossiped that haven't been able to piggyback on probing messages. If this is set to zero, non-piggyback gossip is disabled. By lowering this value (more frequent) gossip messages are propagated across the cluster more quickly at the expense of increased bandwidth. The default is 200ms.

  • probe_interval - The interval between random node probes. Setting this lower (more frequent) will cause the cluster to detect failed nodes more quickly at the expense of increased bandwidth usage. The default is 1s.

  • probe_timeout - The timeout to wait for an ack from a probed node before assuming it is unhealthy. This should be at least the 99-percentile of RTT (round-trip time) on your network. The default is 500ms and is a conservative value suitable for almost all realistic deployments.

  • retransmit_mult - The multiplier for the number of retransmissions that are attempted for messages broadcasted over gossip. The number of retransmits is scaled using this multiplier and the cluster size. The higher the multiplier, the more likely a failed broadcast is to converge at the expense of increased bandwidth. The default is 4.

  • suspicion_mult - The multiplier for determining the time an inaccessible node is considered suspect before declaring it dead. The timeout is scaled with the cluster size and the probe_interval. This allows the timeout to scale properly with expected propagation delay with a larger cluster size. The higher the multiplier, the longer an inaccessible node is considered part of the cluster before declaring it dead, giving that suspect node more time to refute if it is indeed still alive. The default is 4.

• gossip_wan - (Advanced) This object contains a number of sub-keys which can be set to tune the WAN gossip communications. These are only provided for users running especially large clusters that need fine tuning and are prepared to spend significant effort correctly tuning them for their environment and workload. Tuning these improperly can cause Consul to fail in unexpected ways. The default values are appropriate in almost all deployments.

  • gossip_nodes - The number of random nodes to send gossip messages to per gossip_interval. Increasing this number causes the gossip messages to propagate across the cluster more quickly at the expense of increased bandwidth. The default is 3.

  • gossip_interval - The interval between sending messages that need to be gossiped that haven't been able to piggyback on probing messages. If this is set to zero, non-piggyback gossip is disabled. By lowering this value (more frequent) gossip messages are propagated across the cluster more quickly at the expense of increased bandwidth. The default is 200ms.

  • probe_interval - The interval between random node probes. Setting this lower (more frequent) will cause the cluster to detect failed nodes more quickly at the expense of increased bandwidth usage. The default is 1s.

  • probe_timeout - The timeout to wait for an ack from a probed node before assuming it is unhealthy. This should be at least the 99-percentile of RTT (round-trip time) on your network. The default is 500ms and is a conservative value suitable for almost all realistic deployments.

  • retransmit_mult - The multiplier for the number of retransmissions that are attempted for messages broadcasted over gossip. The number of retransmits is scaled using this multiplier and the cluster size. The higher the multiplier, the more likely a failed broadcast is to converge at the expense of increased bandwidth. The default is 4.

  • suspicion_mult - The multiplier for determining the time an inaccessible node is considered suspect before declaring it dead. The timeout is scaled with the cluster size and the probe_interval. This allows the timeout to scale properly with expected propagation delay with a larger cluster size. The higher the multiplier, the longer an inaccessible node is considered part of the cluster before declaring it dead, giving that suspect node more time to refute if it is indeed still alive. The default is 4.

• key_file This provides a the file path to a PEM-encoded private key. The key is used with the certificate to verify the agent's authenticity. This must be provided along with cert_file.

• http_config This object allows setting options for the HTTP API.

  The following sub-keys are available:

  • block_endpoints This object is a list of HTTP API endpoint prefixes to block on the agent, and defaults to an empty list, meaning all endpoints are enabled. Any endpoint that has a common prefix with one of the entries on this list will be blocked and will return a 403 response code when accessed. For example, to block all of the V1 ACL endpoints, set this to ["/v1/acl"], which will block /v1/acl/create, /v1/acl/update, and the other ACL endpoints that begin with /v1/acl. This only works with API endpoints, not /ui or /debug, those must be disabled with their respective configuration options. Any CLI commands that use disabled endpoints will no longer function as well. For more general access control, Consul's ACL system should be used, but this option is useful for removing access to HTTP API endpoints completely, or on specific agents. This is available in Consul 0.9.0 and later.

  • response_headers This object allows adding headers to the HTTP API responses. For example, the following config can be used to enable CORS on the HTTP API endpoints:

    ```javascript
      {
        "http_config": {
          "response_headers": {
            "Access-Control-Allow-Origin": "*"
          }
        }
      }
    ```
    

  • allow_write_http_from This object is a list of networks in CIDR notation (eg "127.0.0.0/8") that are allowed to call the agent write endpoints. It defaults to an empty list, which means all networks are allowed. This is used to make the agent read-only, except for select ip ranges.

    • To block write calls from anywhere, use [ "255.255.255.255/32" ].
    • To only allow write calls from localhost, use [ "127.0.0.0/8" ]
    • To only allow specific IPs, use [ "10.0.0.1/32", "10.0.0.2/32" ]

• leave_on_terminate If enabled, when the agent receives a TERM signal, it will send a Leave message to the rest of the cluster and gracefully leave. The default behavior for this feature varies based on whether or not the agent is running as a client or a server (prior to Consul 0.7 the default value was unconditionally set to false). On agents in client-mode, this defaults to true and for agents in server-mode, this defaults to false.

• limits Available in Consul 0.9.3 and later, this is a nested object that configures limits that are enforced by the agent. Currently, this only applies to agents in client mode, not Consul servers. The following parameters are available:

  • rpc_rate - Configures the RPC rate limiter by setting the maximum request rate that this agent is allowed to make for RPC requests to Consul servers, in requests per second. Defaults to infinite, which disables rate limiting.
  • rpc_max_burst - The size of the token bucket used to recharge the RPC rate limiter. Defaults to 1000 tokens, and each token is good for a single RPC call to a Consul server. See https://en.wikipedia.org/wiki/Token_bucket for more details about how token bucket rate limiters operate.

• log_file Equivalent to the -log-file command-line flag.

• log_rotate_duration Equivalent to the -log-rotate-duration command-line flag.

• log_rotate_bytes Equivalent to the -log-rotate-bytes command-line flag.

• log_rotate_max_files Equivalent to the -log-rotate-max-files command-line flag.

• log_level Equivalent to the -log-level command-line flag.

• node_id Equivalent to the -node-id command-line flag.

• node_name Equivalent to the -node command-line flag.

• node_meta Available in Consul 0.7.3 and later, This object allows associating arbitrary metadata key/value pairs with the local node, which can then be used for filtering results from certain catalog endpoints. See the -node-meta command-line flag for more information.

    {
      "node_meta": {
          "instance_type": "t2.medium"
      }
    }
  

• performance Available in Consul 0.7 and later, this is a nested object that allows tuning the performance of different subsystems in Consul. See the Server Performance documentation for more details. The following parameters are available:

  • leave_drain_time - A duration that a server will dwell during a graceful leave in order to allow requests to be retried against other Consul servers. Under normal circumstances, this can prevent clients from experiencing "no leader" errors when performing a rolling update of the Consul servers. This was added in Consul 1.0. Must be a duration value such as 10s. Defaults to 5s.

  • raft_multiplier - An integer multiplier used by Consul servers to scale key Raft timing parameters. Omitting this value or setting it to 0 uses default timing described below. Lower values are used to tighten timing and increase sensitivity while higher values relax timings and reduce sensitivity. Tuning this affects the time it takes Consul to detect leader failures and to perform leader elections, at the expense of requiring more network and CPU resources for better performance.

    By default, Consul will use a lower-performance timing that's suitable for minimal Consul servers, currently equivalent to setting this to a value of 5 (this default may be changed in future versions of Consul, depending if the target minimum server profile changes). Setting this to a value of 1 will configure Raft to its highest-performance mode, equivalent to the default timing of Consul prior to 0.7, and is recommended for production Consul servers. See the note on last contact timing for more details on tuning this parameter. The maximum allowed value is 10.

  • rpc_hold_timeout - A duration that a client or server will retry internal RPC requests during leader elections. Under normal circumstances, this can prevent clients from experiencing "no leader" errors. This was added in Consul 1.0. Must be a duration value such as 10s. Defaults to 7s.

• ports This is a nested object that allows setting the bind ports for the following keys:

  • dns - The DNS server, -1 to disable. Default 8600. TCP and UDP.
  • http - The HTTP API, -1 to disable. Default 8500. TCP only.
  • https - The HTTPS API, -1 to disable. Default -1 (disabled). We recommend using 8501 for https by convention as some tooling will work automatically with this.
  • grpc - The gRPC API, -1 to disable. Default -1 (disabled). We recommend using 8502 for grpc by convention as some tooling will work automatically with this. This is set to 8502 by default when the agent runs in -dev mode. Currently gRPC is only used to expose Envoy xDS API to Envoy proxies.
  • serf_lan - The Serf LAN port. Default 8301. TCP and UDP.
  • serf_wan - The Serf WAN port. Default 8302. Set to -1 to disable. Note: this will disable WAN federation which is not recommended. Various catalog and WAN related endpoints will return errors or empty results. TCP and UDP.
  • server - Server RPC address. Default 8300. TCP only.
  • sidecar_min_port - Inclusive minimum port number to use for automatically assigned sidecar service registrations. Default 21000. Set to 0 to disable automatic port assignment.
  • sidecar_max_port - Inclusive maximum port number to use for automatically assigned sidecar service registrations. Default 21255. Set to 0 to disable automatic port assignment.
  • expose_min_port - Inclusive minimum port number to use for automatically assigned exposed check listeners. Default 21500. Set to 0 to disable automatic port assignment.
  • expose_max_port - Inclusive maximum port number to use for automatically assigned exposed check listeners. Default 21755. Set to 0 to disable automatic port assignment.

• protocol Equivalent to the -protocol command-line flag.

• primary_datacenter - This designates the datacenter which is authoritative for ACL information, intentions and is the root Certificate Authority for Connect. It must be provided to enable ACLs. All servers and datacenters must agree on the primary datacenter. Setting it on the servers is all you need for cluster-level enforcement, but for the APIs to forward properly from the clients, it must be set on them too. In Consul 0.8 and later, this also enables agent-level enforcement of ACLs.

• raft_protocol Equivalent to the -raft-protocol command-line flag.

• raft_snapshot_threshold This controls the minimum number of raft commit entries between snapshots that are saved to disk. This is a low-level parameter that should rarely need to be changed. Very busy clusters experiencing excessive disk IO may increase this value to reduce disk IO, and minimize the chances of all servers taking snapshots at the same time. Increasing this trades off disk IO for disk space since the log will grow much larger and the space in the raft.db file can't be reclaimed till the next snapshot. Servers may take longer to recover from crashes or failover if this is increased significantly as more logs will need to be replayed. In Consul 1.1.0 and later this defaults to 16384, and in prior versions it was set to 8192.

• raft_snapshot_interval This controls how often servers check if they need to save a snapshot to disk. his is a low-level parameter that should rarely need to be changed. Very busy clusters experiencing excessive disk IO may increase this value to reduce disk IO, and minimize the chances of all servers taking snapshots at the same time. Increasing this trades off disk IO for disk space since the log will grow much larger and the space in th e raft.db file can't be reclaimed till the next snapshot. Servers may take longer to recover from crashes or failover if this is increased significantly as more logs will need to be replayed. In Consul 1.1.0 and later this defaults to 30s, and in prior versions it was set to 5s.

• raft_trailing_logs - This controls how many log entries are left in the log store on disk after a snapshot is made. This should only be adjusted when followers cannot catch up to the leader due to a very large snapshot size that and high write throughput causing log truncation before an snapshot can be fully installed. If you need to use this to recover a cluster, consider reducing write throughput or the amount of data stored on Consul as it is likely under a load it is not designed to handle. The default value is 10000 which is suitable for all normal workloads. Added in Consul 1.5.3.

• reap This controls Consul's automatic reaping of child processes, which is useful if Consul is running as PID 1 in a Docker container. If this isn't specified, then Consul will automatically reap child processes if it detects it is running as PID 1. If this is set to true or false, then it controls reaping regardless of Consul's PID (forces reaping on or off, respectively). This option was removed in Consul 0.7.1. For later versions of Consul, you will need to reap processes using a wrapper, please see the Consul Docker image entry point script for an example. If you are using Docker 1.13.0 or later, you can use the new --init option of the docker run command and docker will enable an init process with PID 1 that reaps child processes for the container. More info on Docker docs.

• reconnect_timeout This controls how long it takes for a failed node to be completely removed from the cluster. This defaults to 72 hours and it is recommended that this is set to at least double the maximum expected recoverable outage time for a node or network partition. WARNING: Setting this time too low could cause Consul servers to be removed from quorum during an extended node failure or partition, which could complicate recovery of the cluster. The value is a time with a unit suffix, which can be "s", "m", "h" for seconds, minutes, or hours. The value must be >= 8 hours.

• reconnect_timeout_wan This is the WAN equivalent of the reconnect_timeout parameter, which controls how long it takes for a failed server to be completely removed from the WAN pool. This also defaults to 72 hours, and must be >= 8 hours.

• recursors This flag provides addresses of upstream DNS servers that are used to recursively resolve queries if they are not inside the service domain for Consul. For example, a node can use Consul directly as a DNS server, and if the record is outside of the "consul." domain, the query will be resolved upstream. As of Consul 1.0.1 recursors can be provided as IP addresses or as go-sockaddr templates. IP addresses are resolved in order, and duplicates are ignored.

• rejoin_after_leave Equivalent to the -rejoin command-line flag.

• retry_join - Equivalent to the -retry-join command-line flag.

• retry_interval Equivalent to the -retry-interval command-line flag.

• retry_join_wan Equivalent to the -retry-join-wan command-line flag. Takes a list of addresses to attempt joining to WAN every retry_interval_wan until at least one join works.

• retry_interval_wan Equivalent to the -retry-interval-wan command-line flag.

• segment (Enterprise-only) Equivalent to the -segment command-line flag.

• segments (Enterprise-only) This is a list of nested objects that allows setting the bind/advertise information for network segments. This can only be set on servers. See the Network Segments Guide for more details.

  • name - The name of the segment. Must be a string between 1 and 64 characters in length.
  • bind - The bind address to use for the segment's gossip layer. Defaults to the -bind value if not provided.
  • port - The port to use for the segment's gossip layer (required).
  • advertise - The advertise address to use for the segment's gossip layer. Defaults to the -advertise value if not provided.
  • rpc_listener - If true, a separate RPC listener will be started on this segment's -bind address on the rpc port. Only valid if the segment's bind address differs from the -bind address. Defaults to false.

• server Equivalent to the -server command-line flag.

• non_voting_server - Equivalent to the -non-voting-server command-line flag.

• server_name When provided, this overrides the node_name for the TLS certificate. It can be used to ensure that the certificate name matches the hostname we declare.

• session_ttl_min The minimum allowed session TTL. This ensures sessions are not created with TTL's shorter than the specified limit. It is recommended to keep this limit at or above the default to encourage clients to send infrequent heartbeats. Defaults to 10s.

• skip_leave_on_interrupt This is similar to leave_on_terminate but only affects interrupt handling. When Consul receives an interrupt signal (such as hitting Control-C in a terminal), Consul will gracefully leave the cluster. Setting this to true disables that behavior. The default behavior for this feature varies based on whether or not the agent is running as a client or a server (prior to Consul 0.7 the default value was unconditionally set to false). On agents in client-mode, this defaults to false and for agents in server-mode, this defaults to true (i.e. Ctrl-C on a server will keep the server in the cluster and therefore quorum, and Ctrl-C on a client will gracefully leave).

• start_join An array of strings specifying addresses of nodes to -join upon startup. Note that using retry_join could be more appropriate to help mitigate node startup race conditions when automating a Consul cluster deployment.

• start_join_wan An array of strings specifying addresses of WAN nodes to -join-wan upon startup.

• telemetry This is a nested object that configures where Consul sends its runtime telemetry, and contains the following keys:

  • circonus_api_token A valid API Token used to create/manage check. If provided, metric management is enabled.

  • circonus_api_app A valid app name associated with the API token. By default, this is set to "consul".

  • circonus_api_url The base URL to use for contacting the Circonus API. By default, this is set to "https://api.circonus.com/v2".

  • circonus_submission_interval The interval at which metrics are submitted to Circonus. By default, this is set to "10s" (ten seconds).

  • circonus_submission_url The check.config.submission_url field, of a Check API object, from a previously created HTTPTRAP check.

  • circonus_check_id The Check ID (not check bundle) from a previously created HTTPTRAP check. The numeric portion of the check._cid field in the Check API object.

  • circonus_check_force_metric_activation Force activation of metrics which already exist and are not currently active. If check management is enabled, the default behavior is to add new metrics as they are encountered. If the metric already exists in the check, it will not be activated. This setting overrides that behavior. By default, this is set to false.

  • circonus_check_instance_id Uniquely identifies the metrics coming from this instance. It can be used to maintain metric continuity with transient or ephemeral instances as they move around within an infrastructure. By default, this is set to hostname:application name (e.g. "host123:consul").

  • circonus_check_search_tag A special tag which, when coupled with the instance id, helps to narrow down the search results when neither a Submission URL or Check ID is provided. By default, this is set to service:application name (e.g. "service:consul").

  • circonus_check_display_name Specifies a name to give a check when it is created. This name is displayed in the Circonus UI Checks list. Available in Consul 0.7.2 and later.

  • circonus_check_tags Comma separated list of additional tags to add to a check when it is created. Available in Consul 0.7.2 and later.

  • circonus_broker_id The ID of a specific Circonus Broker to use when creating a new check. The numeric portion of broker._cid field in a Broker API object. If metric management is enabled and neither a Submission URL nor Check ID is provided, an attempt will be made to search for an existing check using Instance ID and Search Tag. If one is not found, a new HTTPTRAP check will be created. By default, this is not used and a random Enterprise Broker is selected, or the default Circonus Public Broker.

  • circonus_broker_select_tag A special tag which will be used to select a Circonus Broker when a Broker ID is not provided. The best use of this is to as a hint for which broker should be used based on where this particular instance is running (e.g. a specific geo location or datacenter, dc:sfo). By default, this is left blank and not used.

  • disable_hostname This controls whether or not to prepend runtime telemetry with the machine's hostname, defaults to false.

  • dogstatsd_addr This provides the address of a DogStatsD instance in the format host:port. DogStatsD is a protocol-compatible flavor of statsd, with the added ability to decorate metrics with tags and event information. If provided, Consul will send various telemetry information to that instance for aggregation. This can be used to capture runtime information.

  • dogstatsd_tags This provides a list of global tags that will be added to all telemetry packets sent to DogStatsD. It is a list of strings, where each string looks like "my_tag_name:my_tag_value".

  • filter_default This controls whether to allow metrics that have not been specified by the filter. Defaults to true, which will allow all metrics when no filters are provided. When set to false with no filters, no metrics will be sent.

  • metrics_prefix The prefix used while writing all telemetry data. By default, this is set to "consul". This was added in Consul 1.0. For previous versions of Consul, use the config option statsite_prefix in this same structure. This was renamed in Consul 1.0 since this prefix applied to all telemetry providers, not just statsite.

  • prefix_filter This is a list of filter rules to apply for allowing/blocking metrics by prefix in the following format:

    [
      "+consul.raft.apply",
      "-consul.http",
      "+consul.http.GET"
    ]
    

    A leading "+" will enable any metrics with the given prefix, and a leading "-" will block them. If there is overlap between two rules, the more specific rule will take precedence. Blocking will take priority if the same prefix is listed multiple times.

  • prometheus_retention_time If the value is greater than 0s (the default), this enables Prometheus export of metrics. The duration can be expressed using the duration semantics and will aggregates all counters for the duration specified (it might have an impact on Consul's memory usage). A good value for this parameter is at least 2 times the interval of scrape of Prometheus, but you might also put a very high retention time such as a few days (for instance 744h to enable retention to 31 days). Fetching the metrics using prometheus can then be performed using the /v1/agent/metrics?format=prometheus endpoint. The format is compatible natively with prometheus. When running in this mode, it is recommended to also enable the option disable_hostname to avoid having prefixed metrics with hostname. Consul does not use the default Prometheus path, so Prometheus must be configured as follows. Note that using ?format=prometheus in the path won't work as ? will be escaped, so it must be specified as a parameter.

      metrics_path: "/v1/agent/metrics"
      params:
        format: ['prometheus']
    

  • statsd_address This provides the address of a statsd instance in the format host:port. If provided, Consul will send various telemetry information to that instance for aggregation. This can be used to capture runtime information. This sends UDP packets only and can be used with statsd or statsite.

  • statsite_address This provides the address of a statsite instance in the format host:port. If provided, Consul will stream various telemetry information to that instance for aggregation. This can be used to capture runtime information. This streams via TCP and can only be used with statsite.

• syslog_facility When enable_syslog is provided, this controls to which facility messages are sent. By default, LOCAL0 will be used.

• tls_min_version Added in Consul 0.7.4, this specifies the minimum supported version of TLS. Accepted values are "tls10", "tls11" or "tls12". This defaults to "tls12". WARNING: TLS 1.1 and lower are generally considered less secure; avoid using these if possible.

• tls_cipher_suites Added in Consul 0.8.2, this specifies the list of supported ciphersuites as a comma-separated-list. The list of all supported ciphersuites is available in the source code.

• tls_prefer_server_cipher_suites Added in Consul 0.8.2, this will cause Consul to prefer the server's ciphersuite over the client ciphersuites.

• translate_wan_addrs If set to true, Consul will prefer a node's configured WAN address when servicing DNS and HTTP requests for a node in a remote datacenter. This allows the node to be reached within its own datacenter using its local address, and reached from other datacenters using its WAN address, which is useful in hybrid setups with mixed networks. This is disabled by default.

  Starting in Consul 0.7 and later, node addresses in responses to HTTP requests will also prefer a node's configured WAN address when querying for a node in a remote datacenter. An X-Consul-Translate-Addresses header will be present on all responses when translation is enabled to help clients know that the addresses may be translated. The TaggedAddresses field in responses also have a lan address for clients that need knowledge of that address, regardless of translation.

  The following endpoints translate addresses:

  • /v1/catalog/nodes
  • /v1/catalog/node/<node>
  • /v1/catalog/service/<service>
  • /v1/health/service/<service>
  • /v1/query/<query or name>/execute

• ui - Equivalent to the -ui command-line flag.

• ui_dir - Equivalent to the -ui-dir command-line flag. This configuration key is not required as of Consul version 0.7.0 and later. Specifying this configuration key will enable the web UI. There is no need to specify both ui-dir and ui. Specifying both will result in an error.

• unix_sockets - This allows tuning the ownership and permissions of the Unix domain socket files created by Consul. Domain sockets are only used if the HTTP address is configured with the unix:// prefix.

  It is important to note that this option may have different effects on different operating systems. Linux generally observes socket file permissions while many BSD variants ignore permissions on the socket file itself. It is important to test this feature on your specific distribution. This feature is currently not functional on Windows hosts.

  The following options are valid within this construct and apply globally to all sockets created by Consul:

  • user - The name or ID of the user who will own the socket file.
  • group - The group ID ownership of the socket file. This option currently only supports numeric IDs.
  • mode - The permission bits to set on the file.

• verify_incoming

  • If set to true, Consul requires that all incoming connections make use of TLS and that the client provides a certificate signed by a Certificate Authority from the ca_file or ca_path. This applies to both server RPC and to the HTTPS API. By default, this is false, and Consul will not enforce the use of TLS or verify a client's authenticity. Turning on verify_incoming on consul clients protects the HTTPS endpoint, by ensuring that the certificate that is presented by a 3rd party tool to the HTTPS endpoint was created by the CA that the consul client was setup with. If the UI is served, the same checks are performed.

• verify_incoming_rpc - If set to true, Consul requires that all incoming RPC connections make use of TLS and that the client provides a certificate signed by a Certificate Authority from the ca_file or ca_path. By default, this is false, and Consul will not enforce the use of TLS or verify a client's authenticity.

• verify_incoming_https - If set to true, Consul requires that all incoming HTTPS connections make use of TLS and that the client provides a certificate signed by a Certificate Authority from the ca_file or ca_path. By default, this is false, and Consul will not enforce the use of TLS or verify a client's authenticity. To enable the HTTPS API, you must define an HTTPS port via the ports configuration. By default, HTTPS is disabled.

• verify_outgoing - If set to true, Consul requires that all outgoing connections from this agent make use of TLS and that the server provides a certificate that is signed by a Certificate Authority from the ca_file or ca_path. By default, this is false, and Consul will not make use of TLS for outgoing connections. This applies to clients and servers as both will make outgoing connections.

  ~> Security Note: Note that servers that specify verify_outgoing = true will always talk to other servers over TLS, but they still accept non-TLS connections to allow for a transition of all clients to TLS. Currently the only way to enforce that no client can communicate with a server unencrypted is to also enable verify_incoming which requires client certificates too.

• verify_server_hostname - If set to true, Consul verifies for all outgoing TLS connections that the TLS certificate presented by the servers matches server.<datacenter>.<domain> hostname. By default, this is false, and Consul does not verify the hostname of the certificate, only that it is signed by a trusted CA. This setting is critical to prevent a compromised client from being restarted as a server and having all cluster state including all ACL tokens and Connect CA root keys replicated to it. This is new in 0.5.1.

  ~> Security Note: From versions 0.5.1 to 1.4.0, due to a bug, setting this flag alone does not imply verify_outgoing and leaves client to server and server to server RPCs unencrypted despite the documentation stating otherwise. See CVE-2018-19653 for more details. For those versions you must also set verify_outgoing = true to ensure encrypted RPC connections.

• watches - Watches is a list of watch specifications which allow an external process to be automatically invoked when a particular data view is updated. See the watch documentation for more detail. Watches can be modified when the configuration is reloaded.

Ports Used

Consul requires up to 6 different ports to work properly, some on TCP, UDP, or both protocols.

Review the required ports table for a list of required ports and their default settings.

Reloadable Configuration

Reloading configuration does not reload all configuration items. The items which are reloaded include:

• Log level
• Checks
• Services
• Watches
• HTTP Client Address
• TLS Configuration
  • Please be aware that this is currently limited to reload a configuration that is already TLS enabled. You cannot enable or disable TLS only with reloading.
• Node Metadata
• Metric Prefix Filter
• Discard Check Output
• RPC rate limiting