The `agent/xds` package implements the streaming `DeltaAggregatedResources` gRPC service used by Envoy to fetch configuration. At the core of the package is [delta.go](https://github.com/hashicorp/consul/blob/main/agent/xds/delta.go), which contains the implementation of the **Incremental ADS** protocol variant. With this variant there is a single stream between Consul and an Envoy proxy, and on that stream we send configuration diffs based on Envoy's current state.
The remainder of this package contains the logic necessary for generating xDS configuration such as Clusters, Endpoints, Listeners, and Routes from snapshots generated by `proxycfg`.
The xDS server authorizes requests by looking at the proxy ID in the request and ensuring the ACL token has `service:write` access to either the destination service (for kind=ConnectProxy), or the gateway service (for other kinds).
that any data in the snapshot was already filtered, which allows this authorization at the
xDS server to only perform a shallow check against the proxy ID.
## Config Generation
The xDS types that Consul supports as of v1.14 are: Clusters, Endpoints, Listeners, and Routes. For each of these resource types there is a corresponding file such as [listeners.go](https://github.com/hashicorp/consul/blob/main/agent/xds/listeners.go). There, the entry-point will take a proxycfg snapshot and generate xDS configuration depending on the kind of proxy being configured. There are diverging paths depending on whether a sidecar is being configured, or a gateway.
## Testing
Testing changes to this package is generally done at two layers:
- Against golden files, where each test case tests against a fixed file containing the JSON representation of an xDS resource.
- In integration tests, which spin up live instances of Consul and Envoy and make assertions against Envoy's metrics or configuration.
### Golden files
Tests against golden files exists in functions with names such as `TestAllResourcesFromSnapshot`, `TestListenersFromSnapshot`, etc. These tests generate xDS configuration from a `proxycfg.ConfigSnapshot`, mimicking how we generate configuration for Envoy.
The primary source for the test snapshots is `proxycfg.TestConfigSnapshot`. This function will construct a snapshot from a list of events by calling `initialize` and `handleUpdate` as we do in production code. You can attach new update events to the snapshot, or override existing events by emitting a replacement event for an existing `CorrelationID`.
When a new test case is added, the corresponding Golden files can be generated using:
```
go test ./agent/xds -update -run TestAllResourcesFromSnapshot
```
The new golden files then must be **manually** inspected to ensure that the Envoy configuration was generated as expected. Tests against golden files do not assert that the configuration works as intended, but rather that it _looks_ as intended.
### Integration tests
#### New consul-container integration tests
TODO
#### Legacy bash-driven integration tests
Updating one of these integration tests may be appropriate if fixing a bug in functionality that is tested there, or making an improvement to functionality tested there. If the new test involves significant modifications to the bash helpers you should consider using adding a `consul-container` integration test instead.
For more information refer to their [documentation](test/integration/connect/envoy/).
## Delta (Incremental) xDS Protocol
Consul's implementation of the incremental xDS protocol exists in the file [delta.go](https://github.com/hashicorp/consul/blob/main/agent/xds/delta.go). The interactions between Envoy follow the general guidance from [Envoy's documentation](https://www.envoyproxy.io/docs/envoy/latest/api-docs/xds_protocol),
The xDS stream is a bidirectional stream, and messages from Envoy can be either a: subscription request for resources, an ACK for resources it successfully stored, or a NACK for resources that it did not store. ACKs and NACKs can be associated with specific messages by tracking a "nonce", which Consul generates and pushes in every response sent to Envoy. Consul will only ever send xDS resources, into the stream and will only send resources if it believes that Envoy does not already have them.
For mock examples of how interactions over the stream can play out, refer to the tests in [delta_test.go](https://github.com/hashicorp/consul/blob/main/agent/xds/delta_test.go).
To achieve the aim of only sending incremental diffs to Envoy there are maps tracking the state of both Envoy and Consul. The [xDSDeltaType](https://github.com/hashicorp/consul/blob/c7ef04c5979dbc311ff3c67b7bf3028a93e8b0f1/agent/xds/delta.go#L459) struct contains this data for each xDS resource type:
*`subscriptions` tracks the names that Envoy is interested in for a given xDS type. For example, if Envoy is subscribed to Routes, then this map would contain the names of the routes that Envoy is interested in receiving updates about.
*`resourceVersions` tracks the hash of resources that Envoy has ACK'd. This way we can avoid sending data that Envoy already has.
*`pendingUpdates` holds a map used for tracking data that has been sent to Envoy but has not been ACK'd yet. Certain xDS types have strict ordering requirements to avoid dropping traffic. Tracking pending updates allows us to block sending another update of the same type or sending resources that depend on a previous one being acknowledged.
