From 94332edf2efdecfceec403daff5dfce3f2538710 Mon Sep 17 00:00:00 2001 From: Paul Banks Date: Thu, 11 Oct 2018 10:44:42 +0100 Subject: [PATCH] [WIP] Initial draft of Sidecar Service and Managed Proxy deprecation docs (#4752) * Initial draft of Sidecar Service and Managed Proxy deprecation docs * Service definition deprecation notices and sidecar service * gRPC and sidecar service config options; Deprecate managed proxy options * Envoy Docs: Basic envoy command; envoy getting started/intro * Remove change that snuck in * Envoy custom config example * Add agent/service API docs; deprecate proxy config endpoint * Misc grep cleanup for managed proxies; capitalize Envoy * Updates to getting started guide * Add missing link * Refactor Envoy guide into a separate guide and add bootstrap reference notes. * Add limitations to Envoy docs; Highlight no fixes for known managed proxy issues on deprecation page; clarify snake cae stuff; Sidecar Service lifecycle --- agent/agent.go | 2 +- website/source/api/agent/connect.html.md | 17 +- website/source/api/agent/service.html.md | 99 ++++- website/source/api/catalog.html.md | 2 +- website/source/api/index.html.md | 23 + website/source/docs/agent/options.html.md | 72 ++- website/source/docs/agent/services.html.md | 130 ++++-- .../commands/_http_api_options_client.html.md | 8 +- .../docs/commands/connect/envoy.html.md.erb | 178 ++++++++ .../docs/commands/connect/proxy.html.md.erb | 40 +- website/source/docs/commands/index.html.md | 32 +- website/source/docs/connect/index.html.md | 17 + website/source/docs/connect/proxies.html.md | 410 +++++------------- website/source/docs/connect/proxies/envoy.md | 270 ++++++++++++ .../docs/connect/proxies/integrate.html.md | 14 +- .../proxies/managed-deprecated.html.md | 284 ++++++++++++ .../docs/connect/proxies/sidecar-service.md | 178 ++++++++ website/source/docs/guides/connect-envoy.md | 258 +++++++++++ .../source/docs/guides/connect-production.md | 35 +- .../docs/guides/consul-containers.html.md | 2 +- .../intro/getting-started/connect.html.md | 115 +++-- website/source/layouts/docs.erb | 13 + 22 files changed, 1742 insertions(+), 457 deletions(-) create mode 100644 website/source/docs/commands/connect/envoy.html.md.erb create mode 100644 website/source/docs/connect/proxies/envoy.md create mode 100644 website/source/docs/connect/proxies/managed-deprecated.html.md create mode 100644 website/source/docs/connect/proxies/sidecar-service.md create mode 100644 website/source/docs/guides/connect-envoy.md diff --git a/agent/agent.go b/agent/agent.go index 585d411a4..57f8cc78d 100644 --- a/agent/agent.go +++ b/agent/agent.go @@ -2386,7 +2386,7 @@ func (a *Agent) addProxyLocked(proxy *structs.ConnectManagedProxy, persist, From return nil } -// addProxyLocked adds a new local Connect Proxy instance to be managed by the agent. +// AddProxy adds a new local Connect Proxy instance to be managed by the agent. // // It REQUIRES that the service that is being proxied is already present in the // local state. Note that this is only used for agent-managed proxies so we can diff --git a/website/source/api/agent/connect.html.md b/website/source/api/agent/connect.html.md index 421aff08d..53711b696 100644 --- a/website/source/api/agent/connect.html.md +++ b/website/source/api/agent/connect.html.md @@ -218,12 +218,14 @@ $ curl \ - `ValidBefore` `(string)` - The time before which the certificate is valid. Used with `ValidAfter` this can determine the validity period of the certificate. -## Managed Proxy Configuration +## Managed Proxy Configuration ([Deprecated](/docs/connect/proxies/managed-deprecated.html)) -This endpoint returns the configuration for a -[managed proxy](/docs/connect/proxies.html). -Ths endpoint is only useful for _managed proxies_ and not relevant -for unmanaged proxies. +This endpoint returns the configuration for a [managed +proxy](/docs/connect/proxies.html). Ths endpoint is only useful for _managed +proxies_ and not relevant for unmanaged proxies. This endpoint will be removed +in a future major release as part of [managed proxy +deprecation].(/docs/connect/proxies/managed-deprecated.html). The equivalent API +for use will all future proxies is the more generic ` Managed proxy configuration is set in the service definition. When Consul starts the managed proxy, it provides the service ID and ACL token. The proxy @@ -242,7 +244,10 @@ The table below shows this endpoint's support for | Blocking Queries | Consistency Modes | Agent Caching | ACL Required | | ---------------- | ----------------- | ------------- | ---------------------------- | -| `YES` | `all` | `none` | `service:write, proxy token` | +| `YES`1| `all` | `none` | `service:write, proxy token` | + +1 Supports [hash-based +blocking](/api/index.html#hash-based-blocking-queries) only. ### Parameters diff --git a/website/source/api/agent/service.html.md b/website/source/api/agent/service.html.md index 3f833879d..253cf9c1e 100644 --- a/website/source/api/agent/service.html.md +++ b/website/source/api/agent/service.html.md @@ -63,6 +63,90 @@ $ curl \ } ``` +## Get Service Configuration + +This endpoint was added in Consul 1.3.0 and returns the full service definition +for a single service instance registered on the local agent. It is used by +[Connect proxies](/docs/connect/proxies.html) to discover the embedded proxy +configuration that was registered with the instance. + +It is important to note that the services known by the agent may be different +from those reported by the catalog. This is usually due to changes being made +while there is no leader elected. The agent performs active +[anti-entropy](/docs/internals/anti-entropy.html), so in most situations +everything will be in sync within a few seconds. + +| Method | Path | Produces | +| ------ | ---------------------------- | -------------------------- | +| `GET` | `/agent/service/:service_id` | `application/json` | + +The table below shows this endpoint's support for +[blocking queries](/api/index.html#blocking-queries), +[consistency modes](/api/index.html#consistency-modes), +[agent caching](/api/index.html#agent-caching), and +[required ACLs](/api/index.html#acls). + +| Blocking Queries | Consistency Modes | Agent Caching | ACL Required | +| ---------------- | ----------------- | ------------- | -------------- | +| `YES`1| `none` | `none` | `service:read` | + +1 Supports [hash-based +blocking](/api/index.html#hash-based-blocking-queries) only. + +### Parameters + +- `service_id` `(string: )` - Specifies the ID of the service to + fetch. This is specified as part of the URL. + +### Sample Request + +```text +$ curl \ + http://127.0.0.1:8500/v1/agent/service/web-sidecar-proxy +``` + +### Sample Response + +```json +{ + "Kind": "connect-proxy", + "ID": "web-sidecar-proxy", + "Service": "web-sidecar-proxy", + "Tags": null, + "Meta": null, + "Port": 18080, + "Address": "", + "Weights": { + "Passing": 1, + "Warning": 1 + }, + "EnableTagOverride": false, + "ContentHash": "4ecd29c7bc647ca8", + "Proxy": { + "DestinationServiceName": "web", + "DestinationServiceID": "web", + "LocalServiceAddress": "127.0.0.1", + "LocalServicePort": 8080, + "Config": { + "foo": "bar" + }, + "Upstreams": [ + { + "DestinationType": "service", + "DestinationName": "db", + "LocalBindPort": 9191 + } + ] + } +} +``` + +The response has the same structure as the [service +definition](/docs/agent/services.html) with one extra field `ContentHash` which +contains the [hash-based blocking +query](/api/index.html#hash-based-blocking-queries) hash for the result. The +same hash is also present in `X-Consul-ContentHash`. + ## Register Service This endpoint adds a new service, with an optional health check, to the local @@ -126,8 +210,7 @@ service definition keys for compatibility with the config file format. - `Proxy` `(Proxy: nil)` - From 1.2.3 on, specifies the configuration for a Connect proxy instance. This is only valid if `Kind == "connect-proxy"`. See - the [Unmanaged Proxy](/docs/connect/proxies.html#unmanaged-proxies) - documentation for full details. + the [Proxy documentation](/docs/connect/proxies.html) for full details. - `Connect` `(Connect: nil)` - Specifies the [configuration for Connect](/docs/connect/configuration.html). See the @@ -178,10 +261,14 @@ For the `Connect` field, the parameters are: the [Connect](/docs/connect/index.html) protocol [natively](/docs/connect/native.html). If this is true, then Connect proxies, DNS queries, etc. will be able to service discover this service. -- `Proxy` `(Proxy: nil)` - Specifies that a managed Connect proxy should be - started for this service instance, and optionally provides configuration for - the proxy. The format is as documented in - [Managed Proxies](/docs/connect/proxies.html#managed-proxies) . +- `Proxy` `(Proxy: nil)` - + [**Deprecated**](/docs/connect/proxies/managed-deprecated.html) Specifies that + a managed Connect proxy should be started for this service instance, and + optionally provides configuration for the proxy. The format is as documented + in [Managed Proxy Deprecation](/docs/connect/proxies/managed-deprecated.html). +- `SidecarService` `(ServiceDefinition: nil)` - Specifies an optional nested + service definition to register. For more information see + [Sidecar Service Registration](/docs/connect/proxies/sidecar-service.html). ### Sample Payload diff --git a/website/source/api/catalog.html.md b/website/source/api/catalog.html.md index 75f5344e8..618f2d9a0 100644 --- a/website/source/api/catalog.html.md +++ b/website/source/api/catalog.html.md @@ -506,7 +506,7 @@ $ curl \ removed in a future major version release. - `ServiceProxy` is the proxy config as specified in -[Unmanaged Proxies](/docs/connect/proxies.html#unmanaged-proxies). +[Connect Proxies](/docs/connect/proxies.html). - `ServiceConnect` are the [Connect](/docs/connect/index.html) settings. The value of this struct is equivalent to the `Connect` field for service diff --git a/website/source/api/index.html.md b/website/source/api/index.html.md index c529fed89..8149870bc 100644 --- a/website/source/api/index.html.md +++ b/website/source/api/index.html.md @@ -77,6 +77,29 @@ to the supplied maximum `wait` time to spread out the wake up time of any concurrent requests. This adds up to `wait / 16` additional time to the maximum duration. +### Hash-based Blocking Queries + +A limited number of agent endpoints also support blocking however because the +state is local to the agent and not managed with a consistent raft index, their +blocking mechanism is different. + +Since there is no monotonically increasing index, each response instead contains +a header `X-Consul-ContentHash` which is an opaque hash digest generated by +hashing over all fields in the response that are relevant. + +Subsequent requests may be sent with a query parameter `hash=` where +`value` is the last hash header value seen, and this will block until the `wait` +timeout is passed or until the local agent's state changes in such a way that +the hash would be different. + +Other than the different header and query parameter names, the biggest +difference is that hash values are opaque and can't be compared to see if one +result is older or newer than another. In general hash-based blocking will not +return too early due to an idempotent update since the hash will remain the same +unless the result actually changes, however as with index-based blocking there +is no strict guarantee that clients will never observe the same result delivered +before the full timeout has elapsed. + ## Consistency Modes Most of the read query endpoints support multiple levels of consistency. Since diff --git a/website/source/docs/agent/options.html.md b/website/source/docs/agent/options.html.md index b2cdbe1fa..532e79d19 100644 --- a/website/source/docs/agent/options.html.md +++ b/website/source/docs/agent/options.html.md @@ -172,10 +172,11 @@ will exit with an error at startup. * `-dev` - Enable development server mode. This is useful for quickly starting a Consul agent with all persistence options turned off, enabling an in-memory server which can be used for rapid - prototyping or developing against the API. In this mode, - [Connect is enabled](/docs/connect/configuration.html) and will by default - create a new root CA certificate on startup. This mode is **not** intended for - production use as it does not write any data to disk. + prototyping or developing against the API. In this mode, [Connect is + enabled](/docs/connect/configuration.html) and will by default create a new + root CA certificate on startup. This mode is **not** intended for production + use as it does not write any data to disk. The gRPC port is also defaulted to + `8502` in this mode. * `-disable-host-node-id` - Setting this to true will prevent Consul from using information from the host to generate a deterministic node ID, @@ -216,6 +217,10 @@ will exit with an error at startup. initialized with an encryption key, then the provided key is ignored and a warning will be displayed. +* `-grpc-port` - the gRPC API + port to listen on. Default -1 (gRPC disabled). See [ports](#ports) + documentation for more detail. + * `-hcl` - A HCL configuration fragment. This HCL configuration fragment is appended to the configuration and allows to specify the full range of options of a config file on the command line. @@ -225,6 +230,7 @@ will exit with an error at startup. This overrides the default port 8500. This option is very useful when deploying Consul to an environment which communicates the HTTP port through the environment e.g. PaaS like CloudFoundry, allowing you to set the port directly via a Procfile. + * `-log-file` - to redirect all the Consul agent log messages to a file. This can be specified with the complete path along with the name of the log. In case the path doesn't have the filename, the filename defaults to Consul-timestamp.log . Can be combined with -log-rotate-bytes and -log-rotate-duration for a fine-grained log rotation experience. * `-log-rotate-bytes` - to specify the number of bytes that should be written to a log before it needs to be rotated. Unless specified, there is no limit to the number of bytes that can be written to a log file. @@ -477,7 +483,7 @@ definitions support being updated during a reload. "https": "0.0.0.0" }, "ports": { - "https": 8080 + "https": 8501 }, "key_file": "/etc/pki/tls/private/my.key", "cert_file": "/etc/pki/tls/certs/my.crt", @@ -489,11 +495,13 @@ See, especially, the use of the `ports` setting: ```javascript "ports": { - "https": 8080 + "https": 8501 } ``` -Consul will not enable TLS for the HTTP API unless the `https` port has been assigned a port number `> 0`. +Consul will not enable TLS for the HTTP API unless the `https` port has been +assigned a port number `> 0`. We recommend using `8501` for `https` as this +default will automatically work with some tooling. #### Configuration Key Reference @@ -586,24 +594,27 @@ Consul will not enable TLS for the HTTP API unless the `https` port has been ass addresses to bind to, or a [go-sockaddr](https://godoc.org/github.com/hashicorp/go-sockaddr/template) template that can potentially resolve to multiple addresses. - `http` supports binding to a Unix domain socket. A socket can be - specified in the form `unix:///path/to/socket`. A new domain socket will be - created at the given path. If the specified file path already exists, Consul - will attempt to clear the file and create the domain socket in its place. The - permissions of the socket file are tunable via the [`unix_sockets` config construct](#unix_sockets). + `http`, `https` and `grpc` all support binding to a Unix domain socket. A + socket can be specified in the form `unix:///path/to/socket`. A new domain + socket will be created at the given path. If the specified file path already + exists, Consul will attempt to clear the file and create the domain socket + in its place. The permissions of the socket file are tunable via the + [`unix_sockets` config construct](#unix_sockets). When running Consul agent commands against Unix socket interfaces, use the `-http-addr` argument to specify the path to the socket. You can also place the desired values in the `CONSUL_HTTP_ADDR` environment variable. - For TCP addresses, the variable values should be an IP address with the port. For - example: `10.0.0.1:8500` and not `10.0.0.1`. However, ports are set separately in the - `ports` structure when defining them in a configuration file. + For TCP addresses, the environment variable value should be an IP address + _with the port_. For example: `10.0.0.1:8500` and not `10.0.0.1`. However, + ports are set separately in the `ports` structure when + defining them in a configuration file. The following keys are valid: - `dns` - The DNS server. Defaults to `client_addr` - `http` - The HTTP API. Defaults to `client_addr` - `https` - The HTTPS API. Defaults to `client_addr` + - `grpc` - The gRPC API. Defaults to `client_addr` * `advertise_addr` Equivalent to the [`-advertise` command-line flag](#_advertise). @@ -748,13 +759,13 @@ Consul will not enable TLS for the HTTP API unless the `https` port has been ass has been inactive (rotated out) for more than twice the *current* `leaf_cert_ttl`, it will be removed from the trusted list. - * `proxy` This object allows setting options for the Connect proxies. The following sub-keys are available: + * `proxy` [**Deprecated**](/docs/connect/proxies/managed-deprecated.html) This object allows setting options for the Connect proxies. The following sub-keys are available: - * `allow_managed_api_registration` Allows managed proxies to be configured with services that are registered via the Agent HTTP API. Enabling this would allow anyone with permission to register a service to define a command to execute for the proxy. By default, this is false to protect against arbitrary process execution. + * `allow_managed_api_registration` [**Deprecated**](/docs/connect/proxies/managed-deprecated.html) Allows managed proxies to be configured with services that are registered via the Agent HTTP API. Enabling this would allow anyone with permission to register a service to define a command to execute for the proxy. By default, this is false to protect against arbitrary process execution. - * `allow_managed_root` Allows Consul to start managed proxies if Consul is running as root (EUID of the process is zero). We recommend running Consul as a non-root user. By default, this is false to protect inadvertently running external processes as root. + * `allow_managed_root` [**Deprecated**](/docs/connect/proxies/managed-deprecated.html) Allows Consul to start managed proxies if Consul is running as root (EUID of the process is zero). We recommend running Consul as a non-root user. By default, this is false to protect inadvertently running external processes as root. - * `proxy_defaults` This object configures the default proxy settings for [service definitions with managed proxies](/docs/agent/services.html). It accepts the fields `exec_mode`, `daemon_command`, and `config`. These are used as default values for the respective fields in the service definition. + * `proxy_defaults` [**Deprecated**](/docs/connect/proxies/managed-deprecated.html) This object configures the default proxy settings for service definitions with [managed proxies](/docs/connect/proxies/managed-deprecated.html) (now deprecated). It accepts the fields `exec_mode`, `daemon_command`, and `config`. These are used as default values for the respective fields in the service definition. * `replication_token` When provided, this will enable Connect replication using this token to retrieve and replicate the Intentions to the non-authoritative local datacenter. @@ -1134,14 +1145,31 @@ Consul will not enable TLS for the HTTP API unless the `https` port has been ass the bind ports for the following keys: * `dns` - The DNS server, -1 to disable. Default 8600. * `http` - The HTTP API, -1 to disable. Default 8500. - * `https` - The HTTPS API, -1 to disable. Default -1 (disabled). + * `https` - The HTTPS + API, -1 to disable. Default -1 (disabled). **We recommend using `8501`** for + `https` by convention as some tooling will work automatically with this. + * `grpc` - The gRPC API, -1 + to disable. Default -1 (disabled). **We recommend using `8502`** for + `grpc` by convention as some tooling will work automatically with this. + This is set to `8502` by default when the agent runs in `-dev` mode. + Currently gRPC is only used to expose Envoy xDS API to Envoy proxies. * `serf_lan` - The Serf LAN port. Default 8301. * `serf_wan` - The Serf WAN port. Default 8302. Set to -1 to disable. **Note**: this will disable WAN federation which is not recommended. Various catalog and WAN related endpoints will return errors or empty results. * `server` - Server RPC address. Default 8300. - * `proxy_min_port` - Minimum port number to use for automatically assigned [managed Connect proxies](/docs/connect/proxies.html). If Connect is disabled, managed proxies are unused, or ports are always specified, then this value is unused. Defaults to 20000. - * `proxy_max_port` - Maximum port number to use for automatically assigned [managed Connect proxies](/docs/connect/proxies.html). See [`proxy_min_port`](#proxy_min_port) for more information. Defaults to 20255. + * `proxy_min_port` [**Deprecated**](/docs/connect/proxies/managed-deprecated.html) - Minimum port number to use for automatically assigned [managed proxies](/docs/connect/proxies/managed-deprecated.html). Default 20000. + * `proxy_max_port` [**Deprecated**](/docs/connect/proxies/managed-deprecated.html) - Maximum port number to use for automatically assigned [managed proxies](/docs/connect/proxies/managed-deprecated.html). Default 20255. + * `sidecar_min_port` - Inclusive minimum port + number to use for automatically assigned [sidecar service + registrations](/docs/connect/proxies/sidecar-service.html). Default 21000. + Set to `0` to disable automatic port assignment. + * `sidecar_max_port` - Inclusive maximum port + number to use for automatically assigned [sidecar service + registrations](/docs/connect/proxies/sidecar-service.html). Default 21255. + Set to `0` to disable automatic port assignment. * `protocol` Equivalent to the [`-protocol` command-line flag](#_protocol). diff --git a/website/source/docs/agent/services.html.md b/website/source/docs/agent/services.html.md index d85479352..9133a6f25 100644 --- a/website/source/docs/agent/services.html.md +++ b/website/source/docs/agent/services.html.md @@ -45,7 +45,7 @@ example shows all possible fields, but note that only a few are required. } ], "kind": "connect-proxy", - "proxy_destination": "redis", + "proxy_destination": "redis", // Deprecated "proxy": { "destination_service_name": "redis", "destination_service_id": "redis1", @@ -56,7 +56,8 @@ example shows all possible fields, but note that only a few are required. } "connect": { "native": false, - "proxy": { + "sidecar_service": {} + "proxy": { // Deprecated "command": [], "config": {} } @@ -93,39 +94,65 @@ meta object in node definition. All those meta data can be retrieved individually per instance of the service and all the instances of a given service have their own copy of it. -The `kind` field is used to optionally identify the service as an [unmanaged -Connect proxy](/docs/connect/proxies.html#unmanaged-proxies) instance with the -value `connect-proxy`. For typical non-proxy instances the `kind` field must be -omitted. The `proxy` field is also required for unmanaged proxy registrations -and is only valid if `kind` is `connect-proxy`. The only required `proxy` field -is `destination_service_name`. From version 1.2.0 to 1.3.0 this was specified -using `proxy_destination` which still works but is now deprecated. See the -[unmanaged proxy -configuration](/docs/connect/proxies.html#complete-configuration-example) -documentation for full details. +Services may also contain a `token` field to provide an ACL token. This token is +used for any interaction with the catalog for the service, including +[anti-entropy syncs](/docs/internals/anti-entropy.html) and deregistration. + +The `enable_tag_override` can optionally be specified to disable the +anti-entropy feature for this service. If `enable_tag_override` is set to +`TRUE` then external agents can update this service in the +[catalog](/api/catalog.html) and modify the tags. Subsequent +local sync operations by this agent will ignore the updated tags. For +example, if an external agent modified both the tags and the port for +this service and `enable_tag_override` was set to `TRUE` then after the next +sync cycle the service's port would revert to the original value but the +tags would maintain the updated value. As a counter example: If an +external agent modified both the tags and port for this service and +`enable_tag_override` was set to `FALSE` then after the next sync cycle the +service's port *and* the tags would revert to the original value and all +modifications would be lost. + +It's important to note that this applies only to the locally registered +service. If you have multiple nodes all registering the same service +their `enable_tag_override` configuration and all other service +configuration items are independent of one another. Updating the tags +for the service registered on one node is independent of the same +service (by name) registered on another node. If `enable_tag_override` is +not specified the default value is false. See [anti-entropy +syncs](/docs/internals/anti-entropy.html) for more info. + +For Consul 0.9.3 and earlier you need to use `enableTagOverride`. Consul 1.0 +supports both `enable_tag_override` and `enableTagOverride` but the latter is +deprecated and has been removed as of Consul 1.1. + +### Connect + +The `kind` field is used to optionally identify the service as a [Connect +proxy](/docs/connect/proxies.html) instance with the value `connect-proxy`. For +typical non-proxy instances the `kind` field must be omitted. The `proxy` field +is also required for Connect proxy registrations and is only valid if `kind` is +`connect-proxy`. The only required `proxy` field is `destination_service_name`. +For more detail please see [complete proxy configuration +example](/docs/connect/proxies.html#complete-configuration-example) + +-> **Deprecation Notice:** From version 1.2.0 to 1.3.0, proxy destination was +specified using `proxy_destination` at the top level. This will continue to work +until at least 1.5.0 but it's highly recommended to switch to using +`proxy.destination_service_name`. The `connect` field can be specified to configure [Connect](/docs/connect/index.html) for a service. This field is available in -Consul 1.2 and later. The `native` value can be set to true to advertise the -service as [Connect-native](/docs/connect/native.html). If the `proxy` field is -set (even to an empty object), then this will enable a [managed -proxy](/docs/connect/proxies.html) for the service. The fields within `proxy` -are used to configure the proxy and are specified in the [proxy -docs](/docs/connect/proxies.html). If `native` is true, it is an error to also -specifiy a managed proxy instance. +Consul 1.2.0 and later. The `native` value can be set to true to advertise the +service as [Connect-native](/docs/connect/native.html). The `sidecar_service` +field is an optional nested service definition its behavior and defaults are +described in [Sidecar Service +Registration](/docs/connect/proxies/sidecar-service.html). If `native` is true, +it is an error to also specify a sidecar service registration. -The `weights` field is an optional field to specify the weight of a service in -DNS SRV responses. If this field is not specified, its default value is: -`"weights": {"passing": 1, "warning": 1}`. -When a service is `critical`, it is excluded from DNS responses. -Services with warning checks are in included in responses by default, -but excluded if the optional param `only_passing = true` is present in -agent DNS configuration or `?passing` is used via the API. -When DNS SRV requests are made, the response will include the weights -specified given the state of the service. -This allows some instances to be given higher weight if they have more capacity, -and optionally allows reducing load on services with checks in `warning` status -by giving passing instances a higher weight. +-> **Deprecation Notice:** From version 1.2.0 to 1.3.0 during beta, Connect +supported "Managed" proxies which are specified with the `connect.proxy` field. +[Managed Proxies are deprecated](/docs/connect/proxies/managed-deprecated.html) +and the `connect.proxy` field will be removed in a future major release. ### Checks @@ -136,17 +163,29 @@ the DNS interface as well. If a service is failing its health check or a node has any failing system-level check, the DNS interface will omit that node from any service query. -The check must be of the script, HTTP, TCP or TTL type. If it is a script type, -`args` and `interval` must be provided. If it is a HTTP type, `http` and -`interval` must be provided. If it is a TCP type, `tcp` and `interval` must be -provided. If it is a TTL type, then only `ttl` must be provided. The check name -is automatically generated as `service:`. If there are multiple -service checks registered, the ID will be generated as -`service::` where `` is an incrementing number starting -from `1`. +There are several check types that have differing required options as +[documented here](/docs/agent/checks.html). The check name is automatically +generated as `service:`. If there are multiple service checks +registered, the ID will be generated as `service::` where +`` is an incrementing number starting from `1`. -> **Note:** There is more information about [checks here](/docs/agent/checks.html). +### DNS SRV Weights + +The `weights` field is an optional field to specify the weight of a service in +DNS SRV responses. If this field is not specified, its default value is: +`"weights": {"passing": 1, "warning": 1}`. When a service is `critical`, it is +excluded from DNS responses. Services with warning checks are included in +responses by default, but excluded if the optional param `only_passing = true` +is present in agent DNS configuration or `?passing` is used via the API. + +When DNS SRV requests are made, the response will include the weights specified +given the state of the service. This allows some instances to be given higher +weight if they have more capacity, and optionally allows reducing load on +services with checks in `warning` status by giving passing instances a higher +weight. + ### Enable Tag Override and Anti-Entropy Services may also contain a `token` field to provide an ACL token. This token is @@ -236,7 +275,12 @@ delimit service tags. ## Service Definition Parameter Case For historical reasons Consul's API uses `CamelCased` parameter names in -responses, however it's configuration file syntax borrowed from HCL uses -`snake_case`. For this reason the registration APIs accept both cases for -service definition parameters although APIs will return the listings using -`CamelCase`. \ No newline at end of file +responses, however it's configuration file uses `snake_case` for both HCL and +JSON representations. For this reason the registration _HTTP APIs_ accept both +name styles for service definition parameters although APIs will return the +listings using `CamelCase`. + +Note though that **all config file formats require +`snake_case` fields**. We always document service definition examples using +`snake_case` and JSON since this format works in both config files and API +calls. \ No newline at end of file diff --git a/website/source/docs/commands/_http_api_options_client.html.md b/website/source/docs/commands/_http_api_options_client.html.md index 1082ff098..4c963f7ea 100644 --- a/website/source/docs/commands/_http_api_options_client.html.md +++ b/website/source/docs/commands/_http_api_options_client.html.md @@ -16,9 +16,11 @@ * `-http-addr=` - Address of the Consul agent with the port. This can be an IP address or DNS address, but it must include the port. This can also be specified via the `CONSUL_HTTP_ADDR` environment variable. In Consul 0.8 and - later, the default value is http://127.0.0.1:8500, and https can optionally - be used instead. The scheme can also be set to HTTPS by setting the - environment variable `CONSUL_HTTP_SSL=true`. + later, the default value is http://127.0.0.1:8500, and https can optionally be + used instead. The scheme can also be set to HTTPS by setting the environment + variable `CONSUL_HTTP_SSL=true`. This may be a unix domain socket using + `unix:///path/to/socket` if the [agent is configured to + listen](/docs/agent/options.html#addresses) that way. * `-tls-server-name=` - The server name to use as the SNI host when connecting via TLS. This can also be specified via the `CONSUL_TLS_SERVER_NAME` diff --git a/website/source/docs/commands/connect/envoy.html.md.erb b/website/source/docs/commands/connect/envoy.html.md.erb new file mode 100644 index 000000000..83511afe1 --- /dev/null +++ b/website/source/docs/commands/connect/envoy.html.md.erb @@ -0,0 +1,178 @@ +--- +layout: "docs" +page_title: "Commands: Connect Proxy" +sidebar_current: "docs-commands-connect-envoy" +description: > + The connect proxy subcommand is used to run the built-in mTLS proxy for Connect. +--- + +# Consul Connect Envoy + +Command: `consul connect envoy` + +The connect Envoy command is used to generate a bootstrap configuration for +[Envoy proxy](https://envoyproxy.io) for use with [Consul +Connect](/docs/connect/). + +The default behaviour is to generate the necessary bootstrap configuration for +Envoy based on the environment variables and options provided and by taking to +the local Consul agent. It `exec`s an external Envoy binary with that +configuration leaving the Envoy process running in the foreground. An error is +returned on operating systems other than linux or macOS since Envoy does not +build for other platforms currently. + +If the `-bootstrap` option is specified, the bootstrap config is generated in +the same way and then printed to stdout. This allows it to be redirected to a +file and used with `envoy -c bootstrap.json`. This works on all operating +systems allowing configuration to be generated on a host that Envoy doesn't +build on but then used in a virtualized environment that can run Envoy. + +## Usage + +Usage: `consul connect envoy [options] [-- pass-through options]` + +#### API Options + +The standard API options are used to connect to the local agent to discover the +proxy configuration needed. + + - `-grpc-addr=` - Address of the Consul agent with `grpc` port. This can + be an IP address or DNS address, but it must include the port. This can also + be specified via the CONSUL_GRPC_ADDR environment variable. In Consul 1.3 and + later, the default value is http://127.0.0.1:8502, and https can optionally + be used instead. The scheme can also be set to HTTPS by setting the + environment variable CONSUL_HTTP_SSL=true. This may be a unix domain socket + using `unix:///path/to/socket` if the [agent is configured to + listen](/docs/agent/options.html#addresses) that way. + +-> **Note:** gRPC uses the same TLS + settings as the HTTPS API. If HTTPS is enabled then gRPC will require HTTPS + as well. + + <%= partial "docs/commands/http_api_options_client" %> + +#### Envoy Options + +* `-sidecar-for` - The _ID_ (not name if they differ) of the service instance + this proxy will represent. The target service doesn't need to exist on the + local agent yet but a [sidecar proxy + registration](/docs/connect/proxies.html#sidecar-proxy-fields) with + `proxy.destination_service_id` equal to the passed value must be present. If + multiple proxy registrations targeting the same local service instance are + present the command will error and `-proxy-id` should be used instead. + +* `-proxy-id` - The [proxy + service](/docs/connect/proxies.html#proxy-service-definitions) ID on the + local agent. This must already be present on the local agent. + +-> **Note:** If ACLs are enabled, a token granting `service:write` for the + _target_ service (configured in `proxy.destination_service_name`) must be + passed using the `-token` option or `CONSUL_HTTP_TOKEN` environment variable. + This token authorizes the proxy to obtain TLS certificates representing the + target service. + + * `-envoy-binary` - The full path to a specific Envoy binary to exec. By + default the current `$PATH` is searched for `envoy`. + + * `-admin-bind` - The `host:port` to bind Envoy's admin HTTP API. Default is + `localhost:19000`. Envoy requires that this be enabled. The host part must be + resolvable DNS name or IP address. + + * `-bootstrap` - If present, the command will simply output the generated + bootstrap config to stdout in JSON protobuf form. This can be directed to a + file and used to start Envoy with `envoy -c bootstrap.json`. + +~> **Security Note:** If ACLs are enabled the bootstrap JSON will contain the +ACL token from `-token` or the environment and so should be handled as a secret. +This token grants the identity of any service it has `service:write` permission +for and so can be used to access any upstream service that that service is +allowed to access by [Connect intentions](/docs/connect/intentions.html). + + * `-- [pass-through options]` - Any options given after a double dash are passed + directly through to the `envoy` invocation. See [Envoy's + documentation](https://www.envoyproxy.io/docs) for more details. The command + always specifies `--config-file` and `--v2-config-only` and by default passes + `--disable-hot-restart` see [hot restart](#hot-restart). + +## Examples + +Assume a local service instance is registratered on the local agent with a +sidecar proxy (using the [sidecar service +registration](/docs/connect/proxies/sidecar-service.html) helper) as below. + +```hcl +service { + name = "web" + port = 8080 + connect { sidecar_service {} } +} +``` + +The sidecar Envoy process can be started with. + +```text +$ consul connect envoy -sidecar-for web +``` + +This example assumes that the correct [environment variables](#api-options) are +used to set the local agent connection information and ACL token, or that the +agent is using all-default configuration. + +To pass additional arguments directly to Envoy, for example output logging +level, you can use: + +```text +$ consul connect envoy -sidecar-for web -- -l debug +``` + +To run multiple different proxy instances on the same host, you will +need to use `-admin-bind` on all but one to ensure they don't attempt to bind to +the same port as in the following example. + +```text +$ consul connect envoy -sidecar-for db -admin-bind localhost:19001 +``` + +## Exec Security Details + +The command needs to pass the bootstrap config through to Envoy. Envoy currently +only supports passing this as a file path or passing a whole string on the +command line with `--config-yaml`. Since the bootstrap needs to contain the ACL +token to authorize the proxy, this secret needs careful handling. + +Passing a secret via command option is unacceptable as on many unix systems +these are readable to any user on the host for example via `/proc` or via a +setuid process like `ps`. + +Creating a temporary file is more secure in that it can only be read by the +current user but risks leaving secret material on disk for an unbounded length +of time and in a location that is opaque to the operator. + +To work around these issues, the command currently creates a temporary file and +immediately unlinks it so it can't be read by any other process that doesn't +already have the file descriptor. It then writes the bootstrap JSON, and unsets +the CLOEXEC bit on the file handle so that it remains available to the Envoy +process after exec. Finally it `exec`s Envoy with `--config-file /dev/fd/X` +where `X` is the the file descriptor number of the temp file. + +This ensures that Envoy can read the file without any other normal user process +being able to (assuming they don't have privileged access to /proc). Once the +Envoy process stops, there is no longer any reference to the file to clean up. + +## Envoy Hot Restart + +Envoy supports hot restart which requires simple external coordination. By +default, this command will add `--disable-hot-restart` when it runs Envoy. + +The reason for this default behavior is to make it easy to test and run local +demonstrations with multiple Envoy instances outside of cgroups or network +namespaces. + +To use hot restart, Envoy needs to be started with either the `--restart-epoch` +option. If this command detects that option in the pass-through flags it will +_not_ add `--disable-hot-restart` allowing hot restart to work normally. + +The only difference to note over running Envoy directly is that +`--restart-epoch` must be explicitly set to `0` for the initial launch of the +Envoy instance to avoid disabling hot restart entirely. The official +`hot-restarter.py` always sets this option so should work as recommended. \ No newline at end of file diff --git a/website/source/docs/commands/connect/proxy.html.md.erb b/website/source/docs/commands/connect/proxy.html.md.erb index c9264b4bb..4fe992ecc 100644 --- a/website/source/docs/commands/connect/proxy.html.md.erb +++ b/website/source/docs/commands/connect/proxy.html.md.erb @@ -13,7 +13,7 @@ Command: `consul connect proxy` The connect proxy command is used to run Consul's built-in mTLS proxy for use with Connect. This can be used in production to enable a Connect-unaware application to accept and establish Connect-based connections. This proxy -can also be used in development to establish Connect-based connections. +can also be used in development to connect to Connect-enabled services. ## Usage @@ -27,9 +27,28 @@ Usage: `consul connect proxy [options]` #### Proxy Options +* `-sidecar-for` - The _ID_ (not name if they differ) of the service instance + this proxy will represent. The target service doesn't need to exist on the + local agent yet but a [sidecar proxy + registration](/docs/connect/proxies.html#sidecar-proxy-fields) with + `proxy.destination_service_id` equal to the passed value must be present. If + multiple proxy registrations targeting the same local service instance are + present the command will error and `-proxy-id` should be used instead. + +* `-proxy-id` - The [proxy + service](/docs/connect/proxies.html#proxy-service-definitions) ID on the + local agent. This must already be present on the local agent. + +* `-log-level` - Specifies the log level. + +* `-pprof-addr` - Enable debugging via pprof. Providing a host:port (or just ':port') + enables profiling HTTP endpoints on that address. + * `-service` - Name of the service this proxy is representing. This service doesn't need to actually exist in the Consul catalog, but proper ACL - permissions (`service:write`) are required. + permissions (`service:write`) are required. This and the remaining options can + be used to setup a proxy that is not registered already with local config + [useful for development](/docs/connect/dev.html). * `-upstream` - Upstream service to support connecting to. The format should be 'name:addr', such as 'db:8181'. This will make 'db' available on port 8181. @@ -48,17 +67,12 @@ Usage: `consul connect proxy [options]` proxy available as Connect-capable service in the catalog. This is only useful with `-listen`. -* `-register-id` - Optional ID suffix for the service when `-register` is set - to disambiguate the service ID. By default the service ID is "-proxy" - where `` is the `-service` value. - -* `-log-level` - Specifies the log level. - -* `-pprof-addr` - Enable debugging via pprof. Providing a host:port (or just ':port') - enables profiling HTTP endpoints on that address. - -* `-proxy-id` - The proxy's ID on the local agent. This is only useful to - test the managed proxy mode. +* `-register-id` - Optional ID suffix for the service when `-register` is set to + disambiguate the service ID. By default the service ID is "-proxy" + where `` is the `-service` value. In most cases it is now preferable + to use [`consul services register`](/docs/commands/services/register.html) to + register a fully configured proxy instance rather than specify config and + registration via this command. ## Examples diff --git a/website/source/docs/commands/index.html.md b/website/source/docs/commands/index.html.md index f242ecbf5..5742be6d0 100644 --- a/website/source/docs/commands/index.html.md +++ b/website/source/docs/commands/index.html.md @@ -115,7 +115,7 @@ These environment variables and their purpose are described below: ## `CONSUL_HTTP_ADDR` This is the HTTP API address to the *local* Consul agent -(not the remote server) specified as a URI: +(not the remote server) specified as a URI with optional scheme: ``` CONSUL_HTTP_ADDR=127.0.0.1:8500 @@ -127,6 +127,8 @@ or as a Unix socket path: CONSUL_HTTP_ADDR=unix://var/run/consul_http.sock ``` +If the `https://` scheme is used, `CONSUL_HTTP_SSL` is implied to be true. + ### `CONSUL_HTTP_TOKEN` This is the API access token required when access control lists (ACLs) @@ -155,8 +157,9 @@ CONSUL_HTTP_SSL=true ### `CONSUL_HTTP_SSL_VERIFY` -This is a boolean value (default true) to specify SSL certificate verification; setting this value to `false` is not recommended for production use. Example -for development purposes: +This is a boolean value (default true) to specify SSL certificate verification; +setting this value to `false` is not recommended for production use. Example for +development purposes: ``` CONSUL_HTTP_SSL_VERIFY=false @@ -201,3 +204,26 @@ The server name to use as the SNI host when connecting via TLS. ``` CONSUL_TLS_SERVER_NAME=consulserver.domain ``` + +### `CONSUL_GRPC_ADDR` + +Like [`CONSUL_HTTP_ADDR`](#consul_http_addr) but configures the address the +local agent is listening for gRPC requests. Currently gRPC is only used for +integrating [Envoy proxy](/docs/connect/proxies/envoy.html) and must be [enabled +explicitly](/docs/agent/options.html#grpc_port) in agent configuration. + +``` +CONSUL_GRPC_ADDR=127.0.0.1:8502 +``` + +or as a Unix socket path: + +``` +CONSUL_GRPC_ADDR=unix://var/run/consul_grpc.sock +``` + +If the agent is [configured with TLS +certificates](/docs/agent/encryption.html#rpc-encryption-with-tls), then the +gRPC listener will require TLS and present the same certificate as the https +listener. As with `CONSUL_HTTP_ADDR`, if TLS is enabled either the `https://` +scheme should be used, or `CONSUL_HTTP_SSL` set. \ No newline at end of file diff --git a/website/source/docs/connect/index.html.md b/website/source/docs/connect/index.html.md index b8839c9ce..956ac1eda 100644 --- a/website/source/docs/connect/index.html.md +++ b/website/source/docs/connect/index.html.md @@ -69,6 +69,23 @@ local caching, background updating, and support blocking queries. As a result, most API calls operate on purely local in-memory data and can respond in microseconds. +## Getting Started With Connect + +There are several ways to try Connect in different environments. + + * The [Connect introduction](/intro/getting-started/connect.html) in the + Getting Started guide provides a simple walk through of getting two services + to communicate via Connect using only Consul directly on your local machine. + + * The [Envoy guide](/docs/guides/connect-envoy.html) walks through getting + started with Envoy as a proxy, and uses Docker to run components locally + without installing anything else. + + * The [Kubernetes documentation](/docs/platform/k8s/run.html) shows how to get + from an empty Kubernetes cluster to having Consul installed and Envoy + configured to proxy application traffic automatically using the official helm + chart. + ## Agent Caching and Performance To enable microsecond-speed responses on diff --git a/website/source/docs/connect/proxies.html.md b/website/source/docs/connect/proxies.html.md index b8f73f1cd..7c31f0305 100644 --- a/website/source/docs/connect/proxies.html.md +++ b/website/source/docs/connect/proxies.html.md @@ -17,181 +17,131 @@ connections on a loopback address. This requires all external connections to be established via the Connect protocol to provide authentication and authorization. -Consul supports both _managed_ and _unmanaged_ proxies. A managed proxy -is started, configured, and stopped by Consul. An unmanaged proxy is the -responsibility of the user, like any other Consul service. +-> **Deprecation Note:** Managed Proxies are deprecated as of Consul 1.3. See +[managed proxy deprecation](/docs/connect/proxies/managed-deprecated.html) for +more information. It's strongly recommended to switch to one of the approaches +listed on this page as soon as possible. -## Managed Proxies +## Proxy Service Definitions -Managed proxies are started, configured, and stopped by Consul. They are -enabled via basic configurations within the -[service definition](/docs/agent/services.html). -This is the easiest way to start a proxy and allows Consul users to begin -using Connect with only a small configuration change. +Connect proxies are registered using regular [service +definitions](/docs/agent/services.html). They can be registered both in config +files or via the API just like any other service. -Managed proxies also offer the best security. Managed proxies are given -a unique proxy-specific ACL token that allows read-only access to Connect -information for the specific service the proxy is representing. This ACL -token is more restrictive than can be currently expressed manually in -an ACL policy. +Additionally, to reduce the amount of boilerplate needed for a sidecar proxy, +application service definitions may define inline [sidecar service +registrations](/docs/connect/proxies/sidecar-service.html) which are an +opinionated shorthand for a separate full proxy registration as described here. -The default managed proxy is a basic proxy built-in to Consul and written -in Go. Having a basic built-in proxy allows Consul to have a sane default -with performance that is good enough for most workloads. In some basic -benchmarks, the service-to-service communication over the built-in proxy -could sustain 5 Gbps with sub-millisecond latency. Therefore, -the performance impact of even the basic built-in proxy is minimal. +To function as a Connect proxy, they must be declared as a proxy type and +provide information about the service they represent. -Consul will be integrating with advanced proxies in the near future to support -more complex configurations and higher performance. The configuration below is -all for the built-in proxy. +To declare a service as a proxy, the service definition must contain +the following fields: --> **Security note:** 1.) Managed proxies can only be configured -via agent configuration files. They _cannot_ be registered via the HTTP API. -And 2.) Managed proxies are not started at all if Consul is running as root. -Both of these default configurations help prevent arbitrary process -execution or privilege escalation. This behavior can be configured -[per-agent](/docs/agent/options.html#connect_proxy). + * `kind` (string) must be set to `connect-proxy`. This declares that the + service is a proxy type. -### Lifecycle + * `proxy.destination_service_name` (string) must be set to the service that + this proxy is representing. Note that this replaces `proxy_destination` in + versions 1.2.0 to 1.3.0. -The Consul agent starts managed proxies on demand and supervises them, -restarting them if they crash. The lifecycle of the proxy process is decoupled -from the agent so if the agent crashes or is restarted for an upgrade, the -managed proxy instances will _not_ be stopped. + * `port` must be set so that other Connect services can discover the exact + address for connections. `address` is optional if the service is being + registered against an agent, since it'll inherit the node address. -Note that this behaviour while desirable in production might leave proxy -processes running indefinitely if you manually stop the agent and clear it's -data dir during testing. - -To terminate a managed proxy cleanly you need to deregister the service that -requested it. If the agent is already stopped and will not be restarted again, -you may choose to locate the proxy processes and kill them manually. - -While in `-dev` mode, unless a `-data-dir` is explicitly set, managed proxies -switch to being killed when the agent exits since it can't store state in order -to re-adopt them on restart. - -### Minimal Configuration - -Managed proxies are configured within a -[service definition](/docs/agent/services.html). The simplest possible -managed proxy configuration is an empty configuration. This enables the -default managed proxy and starts a listener for that service: +Minimal Example: ```json { - "service": { - "name": "redis", - "port": 6379, - "connect": { "proxy": {} } - } + "name": "redis-proxy", + "kind": "connect-proxy", + "proxy": { + "destination_service_name": "redis" + }, + "port": 8181 } ``` -The listener is started on random port within the configured Connect -port range. It can be discovered using the -[DNS interface](/docs/agent/dns.html#connect-capable-service-lookups) -or -[Catalog API](#). -In most cases, service-to-service communication is established by -a proxy configured with upstreams (described below), which handle the -discovery transparently. +With this service registered, any Connect clients searching for a +Connect-capable endpoint for "redis" will find this proxy. -### Upstream Configuration +### Sidecar Proxy Fields -To transparently discover and establish Connect-based connections to -dependencies, they must be configured with a static port on the managed -proxy configuration: +Most Connect proxies are deployed as "sidecars" which means they are co-located +with a single service instance which they represent and proxy all inbound +traffic to. In this case the following fields must may also be set: + + * `proxy.destination_service_id` (string) is set to the _id_ (and not the + _name_ if they are different) of the specific service instance that is being + proxied. The proxied service is assumed to be registered on the same agent + although it's not strictly validated to allow for un-coordinated + registrations. + + * `proxy.local_service_port` (string) must specify the port the proxy should use + to connect to the _local_ service instance. + + * `proxy.local_service_address` (string) can be set to override the IP or + hostname the proxy should use to connect to the _local_ service. Defaults to + `127.0.0.1`. + +### Complete Configuration Example + +The following is a complete example showing all the options available when +registering a proxy instance. ```json { - "service": { - "name": "web", - "port": 8080, - "connect": { - "proxy": { - "upstreams": [{ - "destination_name": "redis", - "local_bind_port": 1234 - }] - } - } - } + "name": "redis-proxy", + "kind": "connect-proxy", + "proxy": { + "destination_service_name": "redis", + "destination_service_id": "redis1", + "local_service_address": "127.0.0.1", + "local_service_port": 9090, + "config": {}, + "upstreams": [] + }, + "port": 8181 } ``` -In the example above, -"redis" is configured as an upstream with static port 1234 for service "web". -When a TCP connection is established on port 1234, the proxy -will find Connect-compatible "redis" services via Consul service discovery -and establish a TLS connection identifying as "web". +-> **Deprecation Notice:** From version 1.2.0 to 1.3.0, proxy destination was +specified using `proxy_destination` at the top level. This will continue to work +until at least 1.5.0 but it's highly recommended to switch to using +`proxy.destination_service_name`. -~> **Security:** Any application that can communicate to the configured -static port will be able to masquerade as the source service ("web" in the -example above). You must either trust any loopback access on that port or -use namespacing techniques provided by your operating system. +#### Proxy Parameters --> **Deprecation Note:** versions 1.2.0 to 1.3.0 required specifying `upstreams` -as part of the opaque `config` that is passed to the proxy. However, since -1.3.0, the `upstreams` configuration is now specified directily under the -`proxy` key. Old service definitions using the nested config will continue to -work and have the values copied into the new location. This allows the upstreams -to be registered centrally rather than being part of the local-only config -passed to the proxy instance. + - `destination_service_name` `string: ` - Specifies the _name_ of the + service this instance is proxying. Both side-car and centralized + load-balancing proxies must specify this. It is used during service + discovery to find the correct proxy instances to route to for a given service + name. -For full details of the additional configurable options available when using the -built-in proxy see the [built-in proxy configuration -reference](/docs/connect/configuration.html#built-in-proxy-options). + - `destination_service_id` `string: ` - Specifies the _ID_ of a single + specific service instance that this proxy is representing. This is only valid + for side-car style proxies that run on the same node. It is assumed that the + service instance is registered via the same Consul agent so the ID is unique + and has no node qualifier. This is useful to show in tooling which proxy + instance is a side-car for which application instance and will enable + fine-grained analysis of the metrics coming from the proxy. -### Prepared Query Upstreams + - `local_service_address` `string: ` - Specifies the address a side-car + proxy should attempt to connect to the local application instance on. + Defaults to 127.0.0.1. -The upstream destination may also be a -[prepared query](/api/query.html). -This allows complex service discovery behavior such as connecting to -the nearest neighbor or filtering by tags. + - `local_service_port` `int: ` - Specifies the port a side-car + proxy should attempt to connect to the local application instance on. + Defaults to the port advertised by the service instance identified by + `destination_service_id` if it exists otherwise it may be empty in responses. -For example, given a prepared query named "nearest-redis" that is -configured to route to the nearest Redis instance, an upstream can be -configured to route to this query. In the example below, any TCP connection -to port 1234 will attempt a Connect-based connection to the nearest Redis -service. + - `config` `object: ` - Specifies opaque config JSON that will be + stored and returned along with the service instance from future API calls. -```json -{ - "service": { - "name": "web", - "port": 8080, - "connect": { - "proxy": { - "upstreams": [{ - "destination_name": "redis", - "destination_type": "prepared_query", - "local_bind_port": 1234 - }] - } - } - } -} -``` - --> **Note:** Connect does not currently support cross-datacenter -service communication. Therefore, prepared queries with Connect should -only be used to discover services within a single datacenter. See -[Multi-Datacenter Connect](/docs/connect/index.html#multi-datacenter) for -more information. - -For full details of the additional configurable options available when using the -built-in proxy see the [built-in proxy configuration -reference](/docs/connect/configuration.html#built-in-proxy-options). - -### Dynamic Upstreams - -If an application requires dynamic dependencies that are only available -at runtime, it must currently [natively integrate](/docs/connect/native.html) -with Connect. After natively integrating, the HTTP API or -[DNS interface](/docs/agent/dns.html#connect-capable-service-lookups) -can be used. + - `upstreams` `array: ` - Specifies the upstream services + this proxy should create listeners for. The format is defined in + [Upstream Configuration Reference](#upstream-configuration-reference). ### Upstream Configuration Reference @@ -237,164 +187,18 @@ registrations](/docs/agent/services.html#service-definition-parameter-case). * `config` `object: ` - Specifies opaque configuration options that will be provided to the proxy instance for this specific upstream. Can contain any valid JSON object. This might be used to configure proxy-specific features - like timeouts or retries for the given upstream. See the - [built-in proxy configuration reference](/docs/connect/configuration.html#built-in-proxy-options) - for options available when using the built-in proxy. + like timeouts or retries for the given upstream. See the [built-in proxy + configuration + reference](/docs/connect/configuration.html#built-in-proxy-options) for + options available when using the built-in proxy. If using Envoy as a proxy, + see [Envoy configuration + reference](/docs/connect/configuration.html#envoy-options) -### Custom Managed Proxy -[Custom proxies](/docs/connect/proxies/integrate.html) can also be -configured to run as a managed proxy. To configure custom proxies, specify -an alternate command to execute for the proxy: +### Dynamic Upstreams -```json -{ - "service": { - "name": "web", - "port": 8080, - "connect": { - "proxy": { - "exec_mode": "daemon", - "command": ["/usr/bin/my-proxy", "-flag-example"], - "config": { - "foo": "bar" - } - } - } - } -} -``` - -The `exec_mode` value specifies how the proxy is executed. The only -supported value at this time is "daemon". The command is the binary and -any arguments to execute. -The "daemon" mode expects a proxy to run as a long-running, blocking -process. It should not double-fork into the background. The custom -proxy should retrieve its configuration (such as the port to run on) -via the [custom proxy integration APIs](/docs/connect/proxies/integrate.html). - -The default proxy command can be changed at an agent-global level -in the agent configuration. An example in HCL format is shown below. - -``` -connect { - proxy_defaults { - command = ["/usr/bin/my-proxy"] - } -} -``` - -With this configuration, all services registered without an explicit -proxy command will use `my-proxy` instead of the default built-in proxy. - -The `config` key is an optional opaque JSON object which will be passed through -to the proxy via the proxy configuration endpoint to allow any configuration -options the proxy needs to be specified. See the [built-in proxy -configuration reference](/docs/connect/configuration.html#built-in-proxy-options) -for details of config options that can be passed when using the built-in proxy. - -### Managed Proxy Logs - -Managed proxies have both stdout and stderr captured in log files in the agent's -`data_dir`. They can be found in -`/proxy/logs/-std{err,out}.log`. - -The built-in proxy will inherit it's log level from the agent so if the agent is -configured with `log_level = DEBUG`, a proxy it starts will also output `DEBUG` -level logs showing service discovery, certificate and authorization information. - -~> **Note:** In `-dev` mode there is no `data_dir` unless one is explicitly -configured so logging is disabled. You can access logs by providing the -[`-data-dir`](/docs/agent/options.html#_data_dir) CLI option. If a data dir is -configured, this will also cause proxy processes to stay running when the agent -terminates as described in [Lifecycle](#lifecycle). - -## Unmanaged Proxies - -Unmanaged proxies are regular Consul services that are registered as a -proxy type and declare the service they represent. The proxy process must -be started, configured, and stopped manually by some external process such -as an operator or scheduler. - -To declare a service as a proxy, the service definition must contain -the following fields: - - * `kind` (string) must be set to `connect-proxy`. This declares that the - service is a proxy type. - - * `proxy.destination_service_name` (string) must be set to the service that this - proxy is representing. Note that this replaces `proxy_destination` in - versions 1.2.0 to 1.3.0. - - * `port` must be set so that other Connect services can discover the exact - address for connections. `address` is optional if the service is being - registered against an agent, since it'll inherit the node address. - -Minimal Example: - -```json -{ - "name": "redis-proxy", - "kind": "connect-proxy", - "proxy": { - "destination_service_name": "redis" - }, - "port": 8181 -} -``` - -With this service registered, any Connect proxies searching for a -Connect-capable endpoint for "redis" will find this proxy. - -### Complete Configuration Example - -The following is a complete example showing all the options available when -registering an unmanaged proxy instance. - -```json -{ - "name": "redis-proxy", - "kind": "connect-proxy", - "proxy": { - "destination_service_name": "redis", - "destination_service_id": "redis1", - "local_service_address": "127.0.0.1", - "local_service_port": 9090, - "config": {}, - "upstreams": [] - }, - "port": 8181 -} -``` - -#### Proxy Parameters - - - `destination_service_name` `string: ` - Specifies the _name_ of the - service this instance is proxying. Both side-car and centralized - load-balancing proxies must specify this. It is used during service - discovery to find the correct proxy instances to route to for a given service - name. - - - `destination_service_id` `string: ` - Specifies the _ID_ of a single - specific service instance that this proxy is representing. This is only valid - for side-car style proxies that run on the same node. It is assumed that the - service instance is registered via the same Consul agent so the ID is unique - and has no node qualifier. This is useful to show in tooling which proxy - instance is a side-car for which application instance and will enable - fine-grained analysis of the metrics coming from the proxy. - - - `local_service_address` `string: ` - Specifies the address a side-car - proxy should attempt to connect to the local application instance on. - Defaults to 127.0.0.1. - - - `local_service_port` `int: ` - Specifies the port a side-car - proxy should attempt to connect to the local application instance on. - Defaults to the port advertised by the service instance identified by - `destination_service_id` if it exists otherwise it may be empty in responses. - - - `config` `object: ` - Specifies opaque config JSON that will be - stored and returned along with the service instance from future API calls. - - - `upstreams` `array: ` - Specifies the upstream services - this proxy should create listeners for. The format is defined in - [Upstream Configuration Reference](#upstream-configuration-reference). \ No newline at end of file +If an application requires dynamic dependencies that are only available +at runtime, it must currently [natively integrate](/docs/connect/native.html) +with Connect. After natively integrating, the HTTP API or +[DNS interface](/docs/agent/dns.html#connect-capable-service-lookups) +can be used. \ No newline at end of file diff --git a/website/source/docs/connect/proxies/envoy.md b/website/source/docs/connect/proxies/envoy.md new file mode 100644 index 000000000..54b46e8d5 --- /dev/null +++ b/website/source/docs/connect/proxies/envoy.md @@ -0,0 +1,270 @@ +--- +layout: "docs" +page_title: "Connect - Envoy Integration" +sidebar_current: "docs-connect-proxies-envoy" +description: |- + Consul Connect has first-class support for configuring Envoy proxy. +--- + +# Envoy Integration + +Consul Connect has first class support for using +[Envoy](https://www.envoyproxy.io) as a proxy. Consul configures Envoy by +optionally exposing a gRPC service on the local agent that serves [Envoy's xDS +configuration +API](https://github.com/envoyproxy/data-plane-api/blob/master/XDS_PROTOCOL.md). + +Currently Consul only supports TCP proxying between services, however HTTP and +gRPC features are planned for the near future along with first class ways to +configure them in Consul. + +As an interim solution, [custom Envoy configuration](#custom-configuration) can +be specified in [proxy service definition](/docs/connect/proxies.html) allowing +more powerful features of Envoy to be used. + +## Supported Versions + +Consul's Envoy support was added in version 1.3.0. It has been tested against +Envoy 1.7.1 and 1.8.0. + + +## Getting Started + +To get started with Envoy and see a working example you can follow the [Using +Envoy with Connect](/docs/guides/connect-envoy.html) guide. + +## Limitations + +The following list limitations of the Envoy integration as released in 1.3.0. +All of these are planned to be lifted in the near future. + + * Default Envoy configuration only supports Layer 4 (TCP) proxying. More + [advanced listener configuration](#advanced-listener-configuration) is + possible but experimental and requires deep Envoy knowledge. First class + workflows for configuring Layer 7 features across the cluster are planned for + the near future. + * There is currently no way to override the configuration of upstream clusters + which makes it impossible to configure Envoy features like circuit breakers, + load balancing policy, custom protocol settings etc. This will be fixed in a + near-future release first with an "escape hatch" similar to the one for + listeners below, then later with first-class support. + * The configuration delivered to Envoy is suitable for a sidecar proxy + currently. Later we plan to support more flexibility to be able to configure + Envoy as an edge router or gateway and similar. + * There is currently no way to disable the public listener and have a "client + only" sidecar for services that don't expose Connect-enabled service but want + to consume others. This will be fixed in a near-future release. + * Once authorized, a persistent TCP connection will not be closed if the + intentions change to deny access. This is currently a limitation of how TCP + proxy and network authz filter work in Envoy. All new connections will be + denied though and destination services can limit exposure by closing inbound + connections periodically or by a rolling restart of the destination service + as an emergency measure. + +## Bootstrap Configuration + +Envoy requires an initial bootstrap configuration that directs it to the local +agent for further configuration discovery. + +To assist in generating this, Consul 1.3.0 adds a [`consul connect envoy` +command](/docs/commands/connect/envoy.html). The command can either output the +bootstrap configuration directly or can generate it and then `exec` the Envoy +binary as a convenience wrapper. + +Some Envoy configuration options like metrics and tracing sinks can only be +specified via the bootstrap config currently and so a custom bootstrap must be +used. In order to work with Connect it's necessary to start with the following +basic template and add additional configuration as needed. + +```yaml +admin: + # access_log_path and address are required by Envoy, Consul doesn't care what + # they are set to though and never accesses the admin API. +node: + # cluter is required by Envoy but Consul doesn't use it + cluster: "" +static_resources: + clusters: + # local_agent is the "cluster" used to make further discovery requests for + # config and should point to the gRPC port of the local Consul agent instance. + - name: local_agent + connect_timeout: 1s + type: STATIC + # tls_context is needed if and only if Consul agent TLS is configured + tls_context: + common_tls_context: + validation_context: + trusted_ca: + filename: "" + http2_protocol_options: {} + hosts: + - socket_address: + address: "" + port_value: "" +dynamic_resources: + lds_config: + ads: {} + cds_config: + ads: {} + ads_config: + api_type: GRPC + grpc_services: + initial_metadata: + - key: "x-consul-token" + token: "" + envoy_grpc: + cluster_name: local_agent +``` + +This configures a "cluster" pointing to the local Consul agent and sets that as +the target for discovering all types of dynamic resources. + +~> **Security Note**: The bootstrap configuration must contain the Consul ACL +token authorizing the proxy to identify as the target service. As such it should +be treated as a secret value and handled with care - an attacker with access to +one is able to obtain Connect TLS certificates for the target service and so +access anything that service is authorized to connect to. + +## Advanced Listener Configuration + +Consul 1.3.0 includes initial Envoy support which includes automatic Layer 4 +(TCP) proxying over mTLS, and authorization. Near future versions of Consul will +bring Layer 7 features like HTTP-path-based routing, retries, tracing and more. + +-> **Advanced Topic!** This section covers an optional way of taking almost +complete control of Envoy's listener configuration which is provided as a way to +experiment with advanced integrations ahead of full layer 7 feature support. +While we don't plan to remove the ability to do this in the future, it should be +considered experimental and requires in-depth knowledge of Envoy's configuration +format. + +For advanced users there is an "escape hatch" available in 1.3.0. The +`proxy.config` map in the [proxy service +definition](/docs/connect/proxies.html#proxy-service-definitions) may contain a +special key called `envoy_public_listener_json`. If this is set, it's value must +be a string containing the serialized proto3 JSON encoding of a complete [envoy +listener +config](https://www.envoyproxy.io/docs/envoy/v1.8.0/api-v2/api/v2/lds.proto). +Each upstream listener may also be customized in the same way by adding a +`envoy_listener_json` key to the `config` map of [the upstream +definition](/docs/connect/proxies.html#upstream-configuration-reference). + +The JSON supplied may describe a protobuf `types.Any` message with `@type` set +to `type.googleapis.com/envoy.api.v2.Listener`, or it may be the direct encoding +of the listener with no `@type` field. + +Once parsed, it is passed to Envoy in place of the listener config that Consul +would typically configure. The only modifications Consul will make to the config +provided are noted below. + +#### Public Listener Configuration + +For the `proxy.config.envoy_public_listener_json`, every `FilterChain` added to +the listener will have it's `TlsContext` overwritten with the Connect TLS +certificates. This means there is no way to override Connect TLS settings or the +requirement for all inbound clients to present valid Connect certificates. + +Also, every `FilterChain` will have the `envoy.ext_authz` filter prepended to +the filters array to ensure that all incoming connections must be authorized +explicitly by the Consul agent based on their presented client certificate. + +To work properly with Consul Connect, the public listener should bind to the +same address in the service definition so it is discoverable. It may also use +the special cluster name `local_app` to forward requests to a single local +instance if the proxy was configured [as a +sidecar](/docs/connect/proxies.html#sidecar-proxy-fields). + +#### Example + +The following example shows a public listener being configured with an http +connection manager. As specified this behaves exactly like the default TCP proxy +filter however it provides metrics on HTTP request volume and response codes. + +If additional config outside of the listener is needed (for example the +top-level `tracing` configuration to send traces to a collecting service), those +currently need to be added to a custom bootstrap. You may generate the default +connect bootstrap with the [`consul connect envoy -bootstrap` +command](/docs/commands/connect/envoy.html) and then add the required additional +resources. + +```text +service { + kind = "connect-proxy" + name = "web-http-aware-proxy" + port = 8080 + proxy { + destination_service_name web + destination_service_id web + config { + envoy_public_listener_json = < **Consul 1.3.0 deprecates Managed Proxies completely.** It's _strongly_ +recommended you do not build anything using Managed proxies and consider using +[sidecar service +registrations](/docs/connect/proxies/sidecar-service.html) instead. + +Even though this was a beta feature, managed proxies will continue to work at +least until Consul 1.5 to prevent disruption to demonstration and +proof-of-concept deployments of Consul Connect. Anyone using managed proxies +though should aim to change their workflow as soon as possible to avoid issues +with a later upgrade. + +While the current functionality will remain present for a few major releases, +**new and known issues will not be fixed**. + +## Deprecation Rationale + +Originally managed proxies traded higher implementation complexity for an easier +"getting started" user experience. After seeing how Connect was investigated and +adopted during beta it because obvious that they were not the best trade off. + +Managed proxies only really helped in local testing or VM-per-service based +models whereas a lot of users jumped straight to containers where they are not +helpful. They also add only targeted fairly basic supervisor features which +meant most people would want to use something else in production for consistency +with other workloads. So the high implementation cost of building robust process +supervision didn't actually benefit most real use-cases. + +Instead of this Connect 1.3.0 introduces the concept of [sidecar service +registrations](/docs/connect/proxies/sidecar-service.html) which +have almost all of the benefits of simpler configuration but without any of the +additional process management complexity. As a result they can be used to +simplify configuration in both container-based and realistic production +supervisor settings. + +## Managed Proxy Documentation + +As the managed proxy features continue to be supported for now, the rest of this +page will document how they work in the interim. + +-> **Deprecation Note:** It's _strongly_ recommended you do not build anything +using Managed proxies and consider using [sidecar service +registrations](/docs/connect/proxies.html/sidecar-service.html) instead. + +Managed proxies are given +a unique proxy-specific ACL token that allows read-only access to Connect +information for the specific service the proxy is representing. This ACL +token is more restrictive than can be currently expressed manually in +an ACL policy. + +The default managed proxy is a basic proxy built-in to Consul and written +in Go. Having a basic built-in proxy allows Consul to have a sane default +with performance that is good enough for most workloads. In some basic +benchmarks, the service-to-service communication over the built-in proxy +could sustain 5 Gbps with sub-millisecond latency. Therefore, +the performance impact of even the basic built-in proxy is minimal. + +Consul will be integrating with advanced proxies in the near future to support +more complex configurations and higher performance. The configuration below is +all for the built-in proxy. + +-> **Security Note:** 1.) Managed proxies can only be configured +via agent configuration files. They _cannot_ be registered via the HTTP API. +And 2.) Managed proxies are not started at all if Consul is running as root. +Both of these default configurations help prevent arbitrary process +execution or privilege escalation. This behavior can be configured +[per-agent](/docs/agent/options.html#connect_proxy). + +### Lifecycle + +The Consul agent starts managed proxies on demand and supervises them, +restarting them if they crash. The lifecycle of the proxy process is decoupled +from the agent so if the agent crashes or is restarted for an upgrade, the +managed proxy instances will _not_ be stopped. + +Note that this behaviour while desirable in production might leave proxy +processes running indefinitely if you manually stop the agent and clear it's +data dir during testing. + +To terminate a managed proxy cleanly you need to deregister the service that +requested it. If the agent is already stopped and will not be restarted again, +you may choose to locate the proxy processes and kill them manually. + +While in `-dev` mode, unless a `-data-dir` is explicitly set, managed proxies +switch to being killed when the agent exits since it can't store state in order +to re-adopt them on restart. + +### Minimal Configuration + +Managed proxies are configured within a +[service definition](/docs/agent/services.html). The simplest possible +managed proxy configuration is an empty configuration. This enables the +default managed proxy and starts a listener for that service: + +```json +{ + "service": { + "name": "redis", + "port": 6379, + "connect": { "proxy": {} } + } +} +``` + +The listener is started on random port within the configured Connect +port range. It can be discovered using the +[DNS interface](/docs/agent/dns.html#connect-capable-service-lookups) +or +[Catalog API](#). +In most cases, service-to-service communication is established by +a proxy configured with upstreams (described below), which handle the +discovery transparently. + +### Upstream Configuration + +To transparently discover and establish Connect-based connections to +dependencies, they must be configured with a static port on the managed +proxy configuration: + +```json +{ + "service": { + "name": "web", + "port": 8080, + "connect": { + "proxy": { + "upstreams": [{ + "destination_name": "redis", + "local_bind_port": 1234 + }] + } + } + } +} +``` + +In the example above, +"redis" is configured as an upstream with static port 1234 for service "web". +When a TCP connection is established on port 1234, the proxy +will find Connect-compatible "redis" services via Consul service discovery +and establish a TLS connection identifying as "web". + +~> **Security:** Any application that can communicate to the configured +static port will be able to masquerade as the source service ("web" in the +example above). You must either trust any loopback access on that port or +use namespacing techniques provided by your operating system. + +-> **Deprecation Note:** versions 1.2.0 to 1.3.0 required specifying `upstreams` +as part of the opaque `config` that is passed to the proxy. However, since +1.3.0, the `upstreams` configuration is now specified directily under the +`proxy` key. Old service definitions using the nested config will continue to +work and have the values copied into the new location. This allows the upstreams +to be registered centrally rather than being part of the local-only config +passed to the proxy instance. + +For full details of the additional configurable options available when using the +built-in proxy see the [built-in proxy configuration +reference](/docs/connect/configuration.html#built-in-proxy-options). + +### Prepared Query Upstreams + +The upstream destination may also be a +[prepared query](/api/query.html). +This allows complex service discovery behavior such as connecting to +the nearest neighbor or filtering by tags. + +For example, given a prepared query named "nearest-redis" that is +configured to route to the nearest Redis instance, an upstream can be +configured to route to this query. In the example below, any TCP connection +to port 1234 will attempt a Connect-based connection to the nearest Redis +service. + +```json +{ + "service": { + "name": "web", + "port": 8080, + "connect": { + "proxy": { + "upstreams": [{ + "destination_name": "redis", + "destination_type": "prepared_query", + "local_bind_port": 1234 + }] + } + } + } +} +``` + +-> **Note:** Connect does not currently support cross-datacenter +service communication. Therefore, prepared queries with Connect should +only be used to discover services within a single datacenter. See +[Multi-Datacenter Connect](/docs/connect/index.html#multi-datacenter) for +more information. + +For full details of the additional configurable options available when using the +built-in proxy see the [built-in proxy configuration +reference](/docs/connect/configuration.html#built-in-proxy-options). + +### Custom Managed Proxy + +[Custom proxies](/docs/connect/proxies/integrate.html) can also be +configured to run as a managed proxy. To configure custom proxies, specify +an alternate command to execute for the proxy: + +```json +{ + "service": { + "name": "web", + "port": 8080, + "connect": { + "proxy": { + "exec_mode": "daemon", + "command": ["/usr/bin/my-proxy", "-flag-example"], + "config": { + "foo": "bar" + } + } + } + } +} +``` + +The `exec_mode` value specifies how the proxy is executed. The only +supported value at this time is "daemon". The command is the binary and +any arguments to execute. +The "daemon" mode expects a proxy to run as a long-running, blocking +process. It should not double-fork into the background. The custom +proxy should retrieve its configuration (such as the port to run on) +via the [custom proxy integration APIs](/docs/connect/proxies/integrate.html). + +The default proxy command can be changed at an agent-global level +in the agent configuration. An example in HCL format is shown below. + +``` +connect { + proxy_defaults { + command = ["/usr/bin/my-proxy"] + } +} +``` + +With this configuration, all services registered without an explicit +proxy command will use `my-proxy` instead of the default built-in proxy. + +The `config` key is an optional opaque JSON object which will be passed through +to the proxy via the proxy configuration endpoint to allow any configuration +options the proxy needs to be specified. See the [built-in proxy +configuration reference](/docs/connect/configuration.html#built-in-proxy-options) +for details of config options that can be passed when using the built-in proxy. + +### Managed Proxy Logs + +Managed proxies have both stdout and stderr captured in log files in the agent's +`data_dir`. They can be found in +`/proxy/logs/-std{err,out}.log`. + +The built-in proxy will inherit it's log level from the agent so if the agent is +configured with `log_level = DEBUG`, a proxy it starts will also output `DEBUG` +level logs showing service discovery, certificate and authorization information. + +~> **Note:** In `-dev` mode there is no `data_dir` unless one is explicitly +configured so logging is disabled. You can access logs by providing the +[`-data-dir`](/docs/agent/options.html#_data_dir) CLI option. If a data dir is +configured, this will also cause proxy processes to stay running when the agent +terminates as described in [Lifecycle](#lifecycle). \ No newline at end of file diff --git a/website/source/docs/connect/proxies/sidecar-service.md b/website/source/docs/connect/proxies/sidecar-service.md new file mode 100644 index 000000000..770054853 --- /dev/null +++ b/website/source/docs/connect/proxies/sidecar-service.md @@ -0,0 +1,178 @@ +--- +layout: "docs" +page_title: "Connect - Sidecar Service Registration" +sidebar_current: "docs-connect-proxies-sidecar-service" +description: |- + Sidecar service registrations provide a convenient shorthand for registering a + sidecar proxy inline with a regular service definition. +--- + +# Sidecar Service Registration + +Connect proxies are typically deployed as "sidecars" that run on the same node +as the single service instance that they handle traffic for. They might be on +the same VM or running as a separate container in the same network namespace. + +To simplify the configuration experience when deploying a sidecar for a service +instance, Consul 1.3 introduced a new field in the Connect block of the [service +definition](/docs/agent/services.html). + +The `connect.sidecar_service` field is a complete nested service definition on +which almost any regular service definition field can be set. The exceptions are +[noted below](#limitations). If used, the service definition is treated +identically to another top-level service definition. The value of the nested +definition is that _all fields are optional_ with some opinionated defaults +applied that make setting up a sidecar proxy much simpler. + +## Minimal Example + +To register a service instance with a sidecar, all that's needed is: + +```json +{ + "name": "web", + "port": 8080, + "connect": { "sidecar_service": {} } +} +``` + +This will register the `web` service as normal, but will also register another +[proxy service](/docs/connect/proxies.html) with defaults values used. + +The above expands out to be equivalent to the following explicit service +definitions: + +```json +{ + "name": "web", + "port": 8080, +} +{ + "name": "web-sidecar-proxy", + "port": 20000, + "kind": "connect-proxy", + "checks": [ + { + "Name": "Connect Sidecar Listening", + "TCP": "127.0.0.1:20000", + "Interval": "10s" + }, + { + "name": "Connect Sidecar Aliasing web", + "alias_service": "web" + } + ], + "proxy": { + "destination_service_name": "db", + "destination_service_id": "db", + "local_service_address": "127.0.0.1", + "local_service_port": 9090, + } +} +``` + +Details on how the defaults are determined are [documented +below](#sidecar-service-defaults). + +-> **Note:** Sidecar service registrations are only a shorthand for registering +multiple services. Consul will not start up or manage the actual proxy processes +for you. + +## Overridden Example + +The following example shows a service definition where some fields are +overridden to customize the proxy configuration. + +```json +{ + "name": "web", + "port": 8080, + "connect": { + "sidecar_service": { + "proxy": { + "upstreams": [ + { + "destination_name": "db", + "local_bind_port": 9191 + } + ], + "config": { + "handshake_timeout_ms": 1000 + } + } + } + } +} +``` + +This example customizes the [proxy +upstreams](/docs/connect/proxies.html#upstream-configuration-reference) +and some [built-in proxy +configuration](/docs/connect/configuration.html#built-in-proxy-options). + +## Sidecar Service Defaults + +The following fields are set by default on a sidecar service registration. With +[the exceptions noted](#limitations) any field may be overridden explicitly in +the `connect.sidecar_service` definition to customize the proxy registration. +The "parent" service refers to the service definition that embeds the sidecar +proxy. + + - `id` - ID defaults to being `-sidecar-proxy`. This can't + be overridden as it is used to [manage the lifecycle](#lifecycle) of the + registration. + - `name` - Defaults to being `-sidecar-proxy`. + - `port` - Defaults to being auto-assigned from a [configurable + range](/docs/agent/options.html#sidecar_min_port) that is + by default `[21000, 21255]`. + - `kind` - Defaults to `connect-proxy`. This can't be overridden currently. + - `check`, `checks` - By default we add a TCP check on the local address and + port for the proxy, and a [service alias + check](/docs/agent/checks.html#alias) for the parent service. If either + `check` or `checks` fields are set, only the provided checks are registered. + - `proxy.destination_service_name` - Defaults to the parent service name. + - `proxy.destination_service_id` - Defaults to the parent service ID. + - `proxy.local_service_address` - Defaults to `127.0.0.1`. + - `proxy.local_service_port` - Defaults to the parent service port. + +## Limitations + +Almost all fields in a [service definition](/docs/agent/services.html) may be +set on the `connect.sidecar_service` except for the following: + + - `id` - Sidecar services get an ID assigned and it is an error to override + this. This ensures the agent can correctly de-register the sidecar service + later when the parent service is removed. + - `kind` - Kind defaults to `connect-proxy` and there is currently no way to + unset this to make the registration be for a regular non-connect-proxy + service. + - `connect.sidecar_service` - Service definitions can't be nested recursively. + - `connect.proxy` - (Deprecated) [Managed + proxies](/docs/connect/proxies/managed-deprecated.html) can't be defined on a + sidecar. + - `connect.native` - Currently the `kind` is fixed to `connect-proxy` and it's + an error to register a `connect-proxy` that is also Connect-native. + +## Lifecycle + +Sidecar service registration is mostly a configuration syntax helper to avoid +adding lots of boiler plate for basic sidecar options, however the agent does +have some specific behavior around their lifecycle that makes them easier to +work with. + +The agent fixes the ID of the sidecar service to be based on the parent +service's ID. This enables the following behavior. + + - A service instance can _only ever have one_ sidecar service registered. + - When re-registering via API or reloading from configuration file: + - If something changes in the nested sidecar service definition, the change + will _update_ the current sidecar registration instead of creating a new + one. + - If a service registration removes the nested `sidecar_service` then the + previously registered sidecar for that service will be deregistered + automatically. + - When reloading the configuration files, if a service definition changes it's + ID, then a new service instance _and_ a new sidecar instance will be + registered. The old ones will be removed since they are no longer found in + the config files. + diff --git a/website/source/docs/guides/connect-envoy.md b/website/source/docs/guides/connect-envoy.md new file mode 100644 index 000000000..825a91992 --- /dev/null +++ b/website/source/docs/guides/connect-envoy.md @@ -0,0 +1,258 @@ +--- +layout: "docs" +page_title: "Using Envoy with Connect" +sidebar_current: "docs-guides-connect-envoy" +description: |- + This guide walks though getting started running Envoy as a Connect Proxy. +--- + +# Using Connect with Envoy Proxy + +Consul Connect has first class support for using +[Envoy](https://www.envoyproxy.io) as a proxy. This guide will walk through a +working example on a local development machine that shows the moving parts. + +For reference documentation on how the integration works and is configured, +please see [Envoy](/docs/connect/proxies/envoy.html). + +## Setup Overview + +This guide will describe how to setup a development-mode Consul server and two +Envoy proxies on a single machine using [Docker](https://www.docker.com/). The +aim is to demonstrate a minimal working setup and the moving parts involved. + +We choose to run in Docker since Envoy is only distributed as a Docker image so +it's the quickest way to get a demo running. The same commands used here will +work in just the same way outside of Docker if you build an Envoy binary +yourself. + +We'll start all containers using Docker's `host` network mode which is not a +realistic simulation of a production setup, but makes the following steps much +simpler. + +We should end up with five containers running: + + 1. The Consul agent + 2. An example TCP `echo` service as a destination + 3. An Envoy sidecar proxy for the `echo` service + 4. An Envoy sidecar proxy for the `client` service + 5. An example `client` service (netcat) + +## Building an Envoy Image + +Starting Envoy requires a bootstrap configuration file that points Envoy to the +local agent for discovering the rest of it's configuration. The Consul binary +includes the [`consul connect envoy` command](/docs/commands/connect/envoy.html) +which can generate the bootstrap configuration for Envoy and optionally run it +directly. + +Envoy's official Docker image can be used with Connect directly however it +requires some additional steps to generate bootstrap configuration and inject it +into the container. + +Instead, we'll use Docker multi-stage builds (added in version 17.05) to make a +local image that has both `envoy` and `consul` binaries. + +We'll create a local Docker image to use that contains both binaries. First +create a `Dockerfile` containing the following: + +```sh +FROM consul:latest +FROM envoyproxy/envoy:v1.8.0 +COPY --from=0 /bin/consul /bin/consul +ENTRYPOINT ["dumb-init", "consul", "connect", "envoy"] +``` + +This takes the Consul binary from the latest release image and copies it into a +new image based on the official Envoy image. + +This can be built locally with: + +```sh +docker build -t consul-envoy . +``` + +We will use the `consul-envoy` image we just made to configure and run Envoy +processes later. + +## Consul Agent Setup + +Next we need a Consul agent. We'll work with a single Consul agent in `-dev` +mode for simplicity. + +-> **Note:** `-dev` mode enables the gRPC server on port 8502 by default. For a +production agent you'll need to [explicitly configure the gRPC +port](/docs/agent/options.html#grpc_port). + +In order to start a proxy instance, a [proxy service +definition](/docs/connect/proxies.html) must exist on the local agent. We'll +create one using the [sidecar service +registration](/docs/connect/proxies/sidecar-service.html) syntax. + +Create a config file called `envoy_demo.hcl` containing the following service +definitions. + +```hcl +services { + name = "client" + port = 8080 + connect { + sidecar_service { + proxy { + upstreams { + destination_name = "echo" + local_bind_port = 9191 + } + } + } + } +} +services { + name = "echo" + port = 9090 + connect { + sidecar_service {} + } +} +``` + +The Consul agent container can now be started with that configuration. + +```sh +$ docker run --rm -d -v$(pwd)/envoy_demo.hcl:/etc/consul/envoy_demo.hcl \ + --network host --name consul-agent consul:latest \ + agent -dev -config-file /etc/consul/envoy_demo.hcl +1c90f7fcc83f5390332d7a4fdda2f1bf74cf62762de9ea2f67cd5a09c0573641 +``` + +Running with `-d` like this puts the container into the background so we can +continue in the same terminal. Log output can be seen using the name we gave. + +```sh +docker logs -f consul-agent +``` + +Note that the Consul agent has registered two services `client` and `echo`, but +also registered two proxies `client-sidecar-proxy` and `echo-sidecar-proxy`. +Next we'll need to run those services and proxies. + +## Running the Echo Service + +Next we'll run the `echo` service. We can use an existing tcp echo utility image +for this. + +Start the echo service on port 9090 as registered before. + +```sh +$ docker run -d --network host abrarov/tcp-echo --port 9090 +1a0b0c569016d00aadc4fc2b2954209b32b510966083f2a9e17d3afc6d185d87 +``` + +## Running the Proxies + +We can now run "sidecar" proxy instances. + +```sh +$ docker run --rm -d --network host --name echo-proxy \ + consul-envoy -sidecar-for echo +3f213a3cf9b7583a194dd0507a31e0188a03fc1b6e165b7f9336b0b1bb2baccb +$ docker run --rm -d --network host --name client-proxy \ + consul-envoy -sidecar-for client -admin-bind localhost:19001 +d8399b54ee0c1f67d729bc4c8b6e624e86d63d2d9225935971bcb4534233012b +``` + +The `-admin-bind` flag on the second proxy command is needed because both +proxies are running on the host network and so can't bind to the same port for +their admin API (which cannot be disabled). + +Again we can see the output using docker logs. To see more verbose information +from Envoy you can add `-- -l debug` to the end of the commands above. This +passes the `-l` (log level) option directly through to Envoy. With debug level +logs you should see the config being delivered to the proxy in the output. + +The [`consul connect envoy` command](/docs/commands/connect/envoy.html) here is +connecting to the local agent, getting the proxy configuration from the proxy +service registration and generating the required Envoy bootstrap configuration +before `exec`ing the envoy binary directly to run it with the generated +configuration. + +Envoy uses the bootstrap configuration to connect to the local agent directly +via gRPC and use it's xDS protocol to retrieve the actual configuration for +listeners, TLS certificates, upstream service instances and so on. The xDS API +allows the Envoy instance to watch for any changes so certificate rotations or +changes to the upstream service instances are immediately sent to the proxy. + +## Running the Client + +Finally, we can see the connectivity by running a dummy "client" service. Rather +than run a full service that itself can listen, we'll simulate the service with +a simple netcat process that will only talk to the `client-sidecar-proxy` Envoy +instance. + +Recall that we configured the `client` sidecar with one declared "upstream" +dependency (the `echo` service). In that declaration we also requested that the +`echo` service should be exposed to the client on local port 9191. + +This configuration causes the `client-sidecar-proxy` to start a TCP proxy +listening on `localhost:9191` and proxying to the `echo` service. Importantly, +the listener will use the correct `client` service mTLS certificate to authorize +the connection. It discovers the IP addresses of instances of the echo service +via Consul service discovery. + +We can now see this working if we run netcat. + +```sh +$ docker run -ti --rm --network host gophernet/netcat localhost 9191 +Hello World! +Hello World! +^C +``` + +## Testing Authorization + +To demonstrate that Connect is controlling authorization for the echo service, +we can add an explicit deny rule. + +```sh +$ docker run -ti --rm --network host consul:latest intention create -deny client echo +Created: client => echo (deny) +``` + +Now, new connections will be denied. Depending on a few factors, netcat may not +see the connection being closed but will not get a response from the service. + +```sh +$ docker run -ti --rm --network host gophernet/netcat localhost 9191 +Hello? +Anyone there? +^C +``` + +-> **Note:** Envoy will not currently re-authenticate already established TCP +connections so if you still have the netcat terminal open from before, that will +still be able to communicate with "echo". _New_ connections should be denied +though. + +Removing the intention restores connectivity. + +``` +$ docker run -ti --rm --network host consul:latest intention delete client echo +Intention deleted. +$ docker run -ti --rm --network host gophernet/netcat localhost 9191 +Hello? +Hello? +^C +``` + +## Summary + +In this guide we walked through getting a minimal working example of two plain +TCP processes communicating over mTLS using Envoy sidecars configured by +Connect. + +For more details on how the Envoy integration works, please see the [Envoy +reference documentation](/docs/connect/proxies/envoy.html). + +To see how to get Consul Connect working in different environments like +Kubernetes see the [Connect Getting +Started](/docs/connect/index.html#getting-started-with-connect) overview. \ No newline at end of file diff --git a/website/source/docs/guides/connect-production.md b/website/source/docs/guides/connect-production.md index 8bb66539a..da029a122 100644 --- a/website/source/docs/guides/connect-production.md +++ b/website/source/docs/guides/connect-production.md @@ -6,7 +6,7 @@ description: |- This guide describes best practices for running Consul Connect in production. --- -## Running Connect in Production +# Running Connect in Production Consul Connect can secure all inter-service communication via mutual TLS. It's designed to work with [minimal configuration out of the @@ -159,22 +159,21 @@ ports. In addition to Consul agent's [communication ports](/docs/agent/options.html#ports) any -[managed proxies](/docs/connect/proxies.html#managed-proxies) will need to have +[proxies](/docs/connect/proxies.html) will need to have ports open to accept incoming connections. -Consul will by default assign them ports from [a configurable -range](/docs/agent/options.html#ports) the default -range is 20000 - 20255. If this feature is used, the agent assumes all ports in -that range are both free to use (no other processes listening on them) and are -exposed in the firewall to accept connections from other service hosts. +If using [sidecar service +registration](/docs/connect/proxies/sidecar-service.html) Consul will by default +assign ports from [a configurable +range](/docs/agent/options.html#sidecar_min_port) the default range is 21000 - +21255. If this feature is used, the agent assumes all ports in that range are +both free to use (no other processes listening on them) and are exposed in the +firewall to accept connections from other service hosts. -Alternatively, managed proxies can have their public ports specified as part of -the [proxy -configuration](/docs/connect/configuration.html#local_bind_port) in the -service definition. It is possible to use this exclusively and prevent -automated port selection by [configuring `proxy_min_port` and -`proxy_max_port`](/docs/agent/options.html#ports) to both be `0`, forcing any -managed proxies to have an explicit port configured. +It is possible to prevent automated port selection by [configuring +`sidecar_min_port` and +`sidecar_max_port`](/docs/agent/options.html#sidecar_min_port) to both be `0`, +forcing any sidecar service registrations to need an explicit port configured. It then becomes the same problem as opening ports necessary for any other application and might be managed by configuration management or a scheduler. @@ -190,13 +189,9 @@ be set. For registration via the API [the token is passed in the request header](/api/index.html#acls) or by using the [Go client configuration](https://godoc.org/github.com/hashicorp/consul/api#Config). -Note that by default API registration will not allow managed proxies to be -configured since it potentially opens a remote execution vulnerability if the -agent API endpoints are publicly accessible. This can be [configured -per-agent](/docs/agent/options.html#connect_proxy). -For examples of service definitions with managed or unmanaged proxies see -[proxies documentation](/docs/connect/proxies.html#managed-proxies). +For examples of proxy service definitions see the [proxy +documentation](/docs/connect/proxies.html). To avoid the overhead of a proxy, applications may [natively integrate](/docs/connect/native.html) with connect. diff --git a/website/source/docs/guides/consul-containers.html.md b/website/source/docs/guides/consul-containers.html.md index 7ecc69123..c1c80f36a 100644 --- a/website/source/docs/guides/consul-containers.html.md +++ b/website/source/docs/guides/consul-containers.html.md @@ -6,7 +6,7 @@ description: |- This guide describes how to run Consul on containers, with Docker as the primary focus. It also describes best practices when running a Consul cluster in production on Docker. --- -## Consul with Containers +# Consul with Containers This guide describes critical aspects of operating a Consul cluster that's run inside containers. It primarily focuses on the Docker container runtime, but the principles largely apply to rkt, oci, and other container runtimes as well. ## Consul Official Docker Image diff --git a/website/source/intro/getting-started/connect.html.md b/website/source/intro/getting-started/connect.html.md index 89b1f980f..2e97a8604 100644 --- a/website/source/intro/getting-started/connect.html.md +++ b/website/source/intro/getting-started/connect.html.md @@ -28,11 +28,10 @@ guide](/docs/guides/connect-production.html) to understand the tradeoffs. ## Starting a Connect-unaware Service -Let's begin by starting a service that is unaware of Connect all. To -keep it simple, let's just use `socat` to start a basic echo service. This -service will accept TCP connections and echo back any data sent to it. If -`socat` isn't installed on your machine, it should be easily available via -a package manager. +Let's begin by starting a service that is unaware of Connect. To keep it simple, +let's just use `socat` to start a basic echo service. This service will accept +TCP connections and echo back any data sent to it. If `socat` isn't installed on +your machine, it should be easily available via a package manager. ```sh $ socat -v tcp-l:8181,fork exec:"/bin/cat" @@ -67,7 +66,7 @@ $ cat < Consul Connect proxy starting... + Configuration mode: Agent API + Sidecar for ID: socat + Proxy ID: socat-sidecar-proxy + +... +``` + ## Connecting to the Service -Next, let's connect to the service. We'll first do this by using the -`consul connect proxy` command directly. This command manually runs a local -proxy that can represent a service. This is a useful tool for development -since it'll let you masquerade as any service (that you have permissions for) -and establish connections to other services via Connect. +Next, let's connect to the service. We'll first do this by using the `consul +connect proxy` command again directly. This time we use the command to configure +and run a local proxy that can represent a service. This is a useful tool for +development since it'll let you masquerade as any service (that you have +permissions for) and establish connections to other services via Connect. The command below starts a proxy representing a service "web". We request an upstream dependency of "socat" (the service we previously registered) @@ -124,10 +137,10 @@ machine is always encrypted. ## Registering a Dependent Service -We previously established a connection by directly running -`consul connect proxy`. Realistically, services need to establish connections +We previously established a connection by directly running `consul connect +proxy` in developer mode. Realistically, services need to establish connections to dependencies over Connect. Let's register a service "web" that registers -"socat" as an upstream dependency: +"socat" as an upstream dependency in it's sidecar registration: ```sh $ cat < Consul Connect proxy starting... + Configuration mode: Agent API + Sidecar for ID: web + Proxy ID: web-sidecar-proxy + +==> Log data will now stream in as it occurs: + + 2018/10/09 12:34:20 [INFO] 127.0.0.1:9191->service:default/socat starting on 127.0.0.1:9191 + 2018/10/09 12:34:20 [INFO] Proxy loaded config and ready to serve + 2018/10/09 12:34:20 [INFO] TLS Identity: spiffe://df34ef6b-5971-ee61-0790-ca8622c3c287.consul/ns/default/dc/dc1/svc/web + 2018/10/09 12:34:20 [INFO] TLS Roots : [Consul CA 7] +``` + +Note in the first log line that the proxy discovered its configuration from the +local agent and setup a local listener on port 9191 that will proxy to the socat +service just as we configured in the sidecar registration. + +You can also see the identity URL from the certificate it loaded from the agent +identifying it as the "web" service and the set of trusted root CAs it knows +about. + -> **Security note:** The Connect security model requires trusting loopback connections when proxies are in use. To further secure this, tools like network namespacing may be used. +We can verify it works by establishing a new connection: + +``` +$ nc 127.0.0.1 9191 +hello +hello +``` + ## Controlling Access with Intentions Intentions are used to define which services may communicate. Our connections @@ -180,9 +229,18 @@ $ nc 127.0.0.1 9191 $ ``` -Try deleting the intention (or updating it to allow) and attempting the -connection again. Intentions allow services to be segmented via a centralized -control plane (Consul). To learn more, read the reference documentation on +Try deleting the intention and attempt the connection again. + +```sh +$ consul intention delete web socat +Intention deleted. +$ nc 127.0.0.1 9191 +hello +hello +``` + +Intentions allow services to be segmented via a centralized control plane +(Consul). To learn more, read the reference documentation on [intentions](/docs/connect/intentions.html). Note that in the current release of Consul, changing intentions will not @@ -190,6 +248,13 @@ affect existing connections. Therefore, you must establish a new connection to see the effects of a changed intention. This will be addressed in the near term in a future version of Consul. +## Discover More Connect + +This quick guide has given a taste of what Connect can do but there is much +more. Take a look at [getting started with +Connect](/docs/connect/index.html#getting-started-with-connect) for more guides +on setting up Connect with Envoy proxy, with Docker and in Kubernetes. + ## Next Steps We've now configured a service on a single agent and used Connect for diff --git a/website/source/layouts/docs.erb b/website/source/layouts/docs.erb index 51414a6d4..244d5c1ca 100644 --- a/website/source/layouts/docs.erb +++ b/website/source/layouts/docs.erb @@ -79,6 +79,9 @@ > proxy + > + envoy + > @@ -269,6 +272,13 @@ > Proxies