open-consul/website/content/docs/connect/native/index.mdx

158 lines
7.2 KiB
Plaintext
Raw Normal View History

---
2020-04-07 18:55:19 +00:00
layout: docs
page_title: Connect - Native Application Integration
description: >-
Applications can natively integrate with the Connect API to support accepting
and establishing connections to other Connect services without the overhead of
a proxy sidecar.
---
# Connect-Native App Integration
~> **Note:** The Native App Integration does not support many of the Connect service
mesh features, and is not under active development.
The [Envoy proxy](/docs/connect/proxies/envoy) should be used for most production
environments.
[docs] Reorganize connect documentation for clarity (#5864) * clarify possibilities for centralized proxy configuration * add line breaks to config entries file * add info about centralized config to built in proxy doc * mondify connect landing page to help with navigation * move internals details to its own page * link fixes and shortening text on main page * put built-in proxy options on its own page * add configuration details for connect * clarify security title and add observability page * reorganize menu * remove observability from configuration section * Update website/source/docs/connect/configuration.html.md Co-Authored-By: Paul Banks <banks@banksco.de> * Update website/source/docs/connect/index.html.md Co-Authored-By: Paul Banks <banks@banksco.de> * Update website/source/docs/agent/config_entries.html.md Co-Authored-By: Paul Banks <banks@banksco.de> * Update website/source/docs/connect/configuration.html.md Co-Authored-By: Paul Banks <banks@banksco.de> * rename connect section to include service mesh * reorganize sections per suggestions from paul * add configuration edits from paul * add internals edits from paul * add observability edits from paul * reorganize pages and menu * Update website/source/docs/connect/configuration.html.md Co-Authored-By: Paul Banks <banks@banksco.de> * menu corrections and edits * incorporate some of pauls comments * incorporate more of pauls comments * Update website/source/docs/connect/configuration.html.md Co-Authored-By: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com> * Update website/source/docs/connect/index.html.md Co-Authored-By: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com> * Update website/source/docs/connect/index.html.md Co-Authored-By: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com> * Update website/source/docs/connect/registration.html.md Co-Authored-By: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com> * incorporate kaitlin and pavanni feedback * add redirect * fix conflicts in index file * Resolve conflicts in index file * correct links for new organization * Update website/source/docs/connect/proxies.html.md Co-Authored-By: Paul Banks <banks@banksco.de> * Update website/source/docs/connect/registration.html.md Co-Authored-By: Paul Banks <banks@banksco.de> * Update website/source/docs/connect/registration.html.md Co-Authored-By: Paul Banks <banks@banksco.de> * Update website/source/docs/connect/registration.html.md Co-Authored-By: Paul Banks <banks@banksco.de> * add title to service registration page
2019-06-14 05:52:50 +00:00
Applications can natively integrate with the Connect API to support accepting
and establishing connections to other Connect services without the overhead of a
2020-04-09 23:46:54 +00:00
[proxy sidecar](/docs/connect/proxies). This option is especially useful
2020-04-06 20:27:35 +00:00
for applications that may be experiencing performance issues with the proxy
sidecar deployment. This page will cover the high-level overview of
integration, registering the service, etc. For language-specific examples, see
[docs] Reorganize connect documentation for clarity (#5864) * clarify possibilities for centralized proxy configuration * add line breaks to config entries file * add info about centralized config to built in proxy doc * mondify connect landing page to help with navigation * move internals details to its own page * link fixes and shortening text on main page * put built-in proxy options on its own page * add configuration details for connect * clarify security title and add observability page * reorganize menu * remove observability from configuration section * Update website/source/docs/connect/configuration.html.md Co-Authored-By: Paul Banks <banks@banksco.de> * Update website/source/docs/connect/index.html.md Co-Authored-By: Paul Banks <banks@banksco.de> * Update website/source/docs/agent/config_entries.html.md Co-Authored-By: Paul Banks <banks@banksco.de> * Update website/source/docs/connect/configuration.html.md Co-Authored-By: Paul Banks <banks@banksco.de> * rename connect section to include service mesh * reorganize sections per suggestions from paul * add configuration edits from paul * add internals edits from paul * add observability edits from paul * reorganize pages and menu * Update website/source/docs/connect/configuration.html.md Co-Authored-By: Paul Banks <banks@banksco.de> * menu corrections and edits * incorporate some of pauls comments * incorporate more of pauls comments * Update website/source/docs/connect/configuration.html.md Co-Authored-By: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com> * Update website/source/docs/connect/index.html.md Co-Authored-By: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com> * Update website/source/docs/connect/index.html.md Co-Authored-By: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com> * Update website/source/docs/connect/registration.html.md Co-Authored-By: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com> * incorporate kaitlin and pavanni feedback * add redirect * fix conflicts in index file * Resolve conflicts in index file * correct links for new organization * Update website/source/docs/connect/proxies.html.md Co-Authored-By: Paul Banks <banks@banksco.de> * Update website/source/docs/connect/registration.html.md Co-Authored-By: Paul Banks <banks@banksco.de> * Update website/source/docs/connect/registration.html.md Co-Authored-By: Paul Banks <banks@banksco.de> * Update website/source/docs/connect/registration.html.md Co-Authored-By: Paul Banks <banks@banksco.de> * add title to service registration page
2019-06-14 05:52:50 +00:00
the sidebar navigation to the left. It is also required if your service uses
relies on a dynamic set of upstream services.
2018-06-06 05:42:59 +00:00
Connect is just basic mutual TLS. This means that almost any application
can easily integrate with Connect. There is no custom protocol in use;
any language that supports TLS can accept and establish Connect-based
connections.
2020-04-09 23:46:54 +00:00
We currently provide an easy-to-use [Go integration](/docs/connect/native/go)
2018-06-06 18:44:10 +00:00
to assist with the getting the proper certificates, verifying connections,
etc. We plan to add helper libraries for other languages in the future.
However, without library support, it is still possible for any major language
to integrate with Connect.
2018-06-06 05:42:59 +00:00
## Overview
The primary work involved in natively integrating with Connect is
2020-04-09 23:46:54 +00:00
[acquiring the proper TLS certificate](/api/agent/connect#service-leaf-certificate),
[verifying TLS certificates](/api/agent/connect#certificate-authority-ca-roots),
and [authorizing inbound connections or requests](/api/connect/intentions#list-matching-intentions).
2018-06-06 18:44:10 +00:00
All of this is done using the Consul HTTP APIs linked above.
2018-06-06 05:42:59 +00:00
An overview of the sequence is shown below. The diagram and the following
details may seem complex, but this is a _regular mutual TLS connection_ with
an API call to verify the incoming client certificate.
![Native Integration Overview](/img/connect-native-overview.png)
2018-06-06 05:42:59 +00:00
-> **Note:** This diagram depicts the simpler networking layer 4 (e.g. TCP) [integration
mechanism](/api/agent/connect#authorize).
2018-06-06 05:42:59 +00:00
Details on the steps are below:
2020-04-06 20:27:35 +00:00
- **Service discovery** - This is normal service discovery using Consul,
a static IP, or any other mechanism. If you're using Consul DNS, the
[`<service>.connect`](/docs/discovery/dns#connect-capable-service-lookups)
2020-04-06 20:27:35 +00:00
syntax to find Connect-capable endpoints for a service. After service
discovery, choose one address from the list of **service addresses**.
- **Mutual TLS** - As a client, connect to the discovered service address
over normal TLS. As part of the TLS connection, provide the
2020-04-09 23:46:54 +00:00
[service certificate](/api/agent/connect#service-leaf-certificate)
2020-04-06 20:27:35 +00:00
as the client certificate. Verify the remote certificate against the
2020-04-09 23:46:54 +00:00
[public CA roots](/api/agent/connect#certificate-authority-ca-roots).
2020-04-06 20:27:35 +00:00
As a client, if the connection is established then you've established
a Connect-based connection and there are no further steps!
- **Authorization** - As a server accepting connections, verify the client
certificate against the [public CA
roots](/api/agent/connect#certificate-authority-ca-roots). After verifying
the certificate, parse some basic fields from it and use those to determine
if the connection should be allowed. How this is done is dependent on
the level of integration desired:
- **Simple integration (TCP-only)** - Call the [authorizing
API](/api/agent/connect#authorize) against the local agent. If this returns
successfully, complete the TLS handshake and establish the connection. If
authorization fails, close the connection.
-> **NOTE:** This API call is expected to be called in the connection path,
so if the local Consul agent is down or unresponsive it will effect the
success rate of new connections. The agent uses locally cached data to
authorize the connection and typically responds in microseconds. Therefore,
the impact to the TLS handshake is typically microseconds.
- **Complete integration** - Like how the calls to acquire the leaf
certificate and CA roots are expected to be done out of band and reused, so
should the [intention match
API](/api/connect/intentions#list-matching-intentions). With all of the
relevant intentions cached for the destination, all enforcement operations
can be done entirely by the service without calling any Consul APIs in the
connection or request path. If the service is networking layer 7 (e.g.
HTTP) aware it can safely enforce intentions per _request_ instead of the
coarser per _connection_ model.
2018-06-06 05:42:59 +00:00
## Updating Certificates and Certificate Roots
The leaf certificate and CA roots can be updated at any time and the
natively integrated application must react to this relatively quickly
so that new connections are not disrupted. This can be done through
Consul blocking queries (HTTP long polling) or through periodic polling.
The API calls for
2020-04-09 23:46:54 +00:00
[acquiring a leaf TLS certificate](/api/agent/connect#service-leaf-certificate)
and [reading CA roots](/api/agent/connect#certificate-authority-ca-roots)
2018-06-06 05:42:59 +00:00
both support
2020-04-09 23:46:54 +00:00
[blocking queries](/api/features/blocking). By using blocking
2018-06-06 05:42:59 +00:00
queries, an application can efficiently wait for an updated value. For example,
the leaf certificate API will block until the certificate is near expiration
or the signing certificates have changed and will issue and return a new
certificate.
In some languages, using blocking queries may not be simple. In that case,
we still recommend using the blocking query parameters but with a very short
`timeout` value set. Doing this is documented with
2020-04-09 23:46:54 +00:00
[blocking queries](/api/features/blocking). The low timeout will
2018-06-06 05:42:59 +00:00
ensure the API responds quickly. We recommend that applications poll the
certificate endpoints frequently, such as multiple times per minute.
The overhead for the blocking queries (long or periodic polling) is minimal.
The API calls are to the local agent and the local agent uses locally
cached data multiplexed over a single TCP connection to the Consul leader.
Even if a single machine has 1,000 Connect-enabled services all blocking
on certificate updates, this translates to only one TCP connection to the
Consul server.
Some language libraries such as the
2020-04-09 23:46:54 +00:00
[Go library](/docs/connect/native/go) automatically handle updating
2018-06-06 05:42:59 +00:00
and locally caching the certificates.
## Service Registration
Connect-native applications must tell Consul that they support Connect
natively. This enables the service to be returned as part of service
discovery for Connect-capable services, used by other Connect-native applications
2020-04-09 23:46:54 +00:00
and client [proxies](/docs/connect/proxies).
2018-06-06 05:42:59 +00:00
2020-04-09 23:46:54 +00:00
This can be specified directly in the [service definition](/docs/agent/services):
2018-06-06 05:42:59 +00:00
```json
{
"service": {
"name": "redis",
"port": 8000,
"connect": {
"native": true
}
}
}
```
Services that support Connect natively are still returned through the standard
service discovery mechanisms in addition to the Connect-only service discovery
mechanisms.