Commit graph

372 commits

Author SHA1 Message Date
freddygv d1d513b1b3 Account for partition in SNI for gateways 2021-10-27 11:15:25 -06:00
freddygv 4f0432be5e Update xds pkg to account for GatewayKey 2021-10-27 09:03:56 -06:00
freddygv 4737ad118d Swap in structs.EqualPartitions for cmp 2021-10-26 23:36:01 -06:00
freddygv 3966677aaf Finish removing useInDatacenter 2021-10-26 23:36:01 -06:00
freddygv 69476221c1 Update XDS for sidecars dialing through gateways 2021-10-26 23:35:48 -06:00
Paul Banks 4808b97d9c Rebase and rebuild golden files for Envoy version bump 2021-10-19 21:37:58 +01:00
Paul Banks ff405d35c7 Refactor resolveListenerSDSConfig to pass in whole config 2021-10-19 20:58:29 +01:00
Paul Banks 5c8702b182 Add support for enabling connect-based ingress TLS per listener. 2021-10-19 20:58:28 +01:00
Giulio Micheloni a5a4eb9cae Separete test file and no stack trace in ret error 2021-10-16 18:02:03 +01:00
Giulio Micheloni 10814d934e Merge branch 'main' of https://github.com/hashicorp/consul into hashicorp-main 2021-10-16 16:59:32 +01:00
Evan Culver e47c5c5ceb
Merge pull request #11118 from hashicorp/eculver/remove-envoy-1.15
Remove support for Envoy 1.15
2021-10-04 23:14:24 +02:00
Evan Culver d279c60010
Merge pull request #11115 from hashicorp/eculver/envoy-1.19.1
Add support for Envoy 1.19.1
2021-10-04 23:13:26 +02:00
Evan Culver e74ce0fb2e
Add 1.15 versions to too old list 2021-10-01 11:28:26 -07:00
Evan Culver 4cdcaf3658
Merge branch 'eculver/envoy-1.19.1' into eculver/remove-envoy-1.15 2021-09-30 11:32:28 -07:00
Evan Culver 7b157bba4e
regenerate more envoy golden files 2021-09-30 10:57:47 -07:00
Daniel Nephin bd28d23b55 command/envoy: stop using the DebugConfig from Self endpoint
The DebugConfig in the self endpoint can change at any time. It's not a stable API.

This commit adds the XDSPort to a stable part of the XDS api, and changes the envoy command to read
this new field.

It includes support for the old API as well, in case a newer CLI is used with an older API, and
adds a test for both cases.
2021-09-29 13:21:28 -04:00
Daniel Nephin 402d3792b6 Revert "Merge pull request #10588 from hashicorp/dnephin/config-fix-ports-grpc"
This reverts commit 74fb650b6b966588f8faeec26935a858af2b8bb5, reversing
changes made to 58bd8173364effb98b9fd9f9b98d31dd887a9bac.
2021-09-29 12:28:41 -04:00
Evan Culver cb5ef13fde
Merge remote-tracking branch 'origin/eculver/remove-envoy-1.15' into eculver/remove-envoy-1.15 2021-09-28 16:06:36 -07:00
Evan Culver eaa9394cb2
Fix typo
Co-authored-by: Freddy <freddygv@users.noreply.github.com>
2021-09-29 01:05:45 +02:00
Evan Culver 64f94b10ce
Merge branch 'eculver/envoy-1.19.1' into eculver/remove-envoy-1.15 2021-09-28 15:59:43 -07:00
Evan Culver e2363c13ff
Merge branch 'main' into eculver/envoy-1.19.1 2021-09-28 11:54:33 -07:00
Paul Banks d57931124f Final readability tweaks from review 2021-09-23 10:17:12 +01:00
Paul Banks 66c625a64d Fix subtle loop bug and add test 2021-09-23 10:13:41 +01:00
Paul Banks fe4f69613c Refactor Ingress-specific lister code to separate file 2021-09-23 10:13:19 +01:00
Paul Banks f4f0793a10 Minor PR typo and cleanup fixes 2021-09-23 10:13:19 +01:00
Paul Banks d812a0edc7 Fix merge conflict in xds tests 2021-09-23 10:12:37 +01:00
Paul Banks 15969327c0 Remove unused argument to fix lint error 2021-09-23 10:09:11 +01:00
Paul Banks 9422e4ebc7 Handle namespaces in route names correctly; add tests for enterprise 2021-09-23 10:09:11 +01:00
Paul Banks 9d576a08dc Update xDS routes to support ingress services with different TLS config 2021-09-23 10:08:02 +01:00
Paul Banks 8a4254a894 Update xDS Listeners with SDS support 2021-09-23 10:08:02 +01:00
Chris S. Kim d222f170a7
connect: Allow upstream listener escape hatch for prepared queries (#11109) 2021-09-22 15:27:10 -04:00
Evan Culver 88a899d06a
connect: remove support for Envoy 1.15 2021-09-22 11:48:50 -07:00
Evan Culver 4d222cfcd0
add 1.19.x versions to test config 2021-09-22 09:30:45 -07:00
Evan Culver 69f4cc7532
regenerate envoy golden files 2021-09-21 16:21:00 -07:00
Evan Culver b104b7719c
add envoy 1.19.1 2021-09-21 15:39:36 -07:00
R.B. Boyer 2773bd94d7
xds: fix representation of incremental xDS subscriptions (#10987)
Fixes #10563

The `resourceVersion` map was doing two jobs prior to this PR. The first job was
to track what version of every resource we know envoy currently has. The
second was to track subscriptions to those resources (by way of the empty
string for a version). This mostly works out fine, but occasionally leads to
consul removing a resource and accidentally (effectively) unsubscribing at the
same time.

The fix separates these two jobs. When all of the resources for a subscription
are removed we continue to track the subscription until envoy explicitly
unsubscribes
2021-09-21 09:58:56 -05:00
R.B. Boyer 55b36dd056
xds: ensure the active streams counters are 64 bit aligned on 32 bit systems (#11085) 2021-09-20 11:07:11 -05:00
freddygv a57c52ca32 Ensure partition is used for SAN validation 2021-09-15 17:23:48 -06:00
freddygv f209408918 Update spiffe ID patterns used for RBAC 2021-09-14 11:00:03 -06:00
freddygv 0e30151eaa Expand testing of simplifyNotSourceSlice for partitions 2021-09-14 10:55:15 -06:00
freddygv a65da57a3d Expand testing of removeSameSourceIntentions for partitions 2021-09-14 10:55:09 -06:00
freddygv e9d78a20c7 Account for partition when matching src intentions 2021-09-14 10:55:02 -06:00
Paul Banks 1dd1683ed9 Header manip for split legs plumbing 2021-09-10 21:09:24 +01:00
Paul Banks f70f7b2389 Header manip for service-router plumbed through 2021-09-10 21:09:24 +01:00
Paul Banks fc2ed4cdf4 Ingress gateway header manip plumbing 2021-09-10 21:09:24 +01:00
Dhia Ayachi 96d7842118
partition dicovery chains (#10983)
* partition dicovery chains

* fix default partition for OSS
2021-09-07 16:29:32 -04:00
Dhia Ayachi eb19271fd7
add partition to SNI when partition is non default (#10917) 2021-09-01 10:35:39 -04:00
Freddy 11672defaf
connect: update envoy supported versions to latest patch release
(#10961)

Relevant advisory: 
https://github.com/envoyproxy/envoy/security/advisories/GHSA-6g4j-5vrw-2m8h
2021-08-31 10:39:18 -06:00
Giulio Micheloni 387f6f717b Fix merge conflicts 2021-08-22 19:35:08 +01:00
Giulio Micheloni 10b03c3f4e
Merge branch 'main' into serve-panic-recovery 2021-08-22 20:31:11 +02:00
Giulio Micheloni 465e9fecda grpc, xds: recovery middleware to return and log error in case of panic
1) xds and grpc servers:
   1.1) to use recovery middleware with callback that prints stack trace to log
   1.2) callback turn the panic into a core.Internal error
2) added unit test for grpc server
2021-08-22 19:06:26 +01:00
freddygv 79e181be73 Avoid passing zero value into variadic 2021-08-20 17:40:33 -06:00
freddygv b1050e4229 Update prepared query cluster SAN validation
Previously SAN validation for prepared queries was broken because we
validated against the name, namespace, and datacenter for prepared
queries.

However, prepared queries can target:

- Services with a name that isn't their own
- Services in multiple datacenters

This means that the SpiffeID to validate needs to be based on the
prepared query endpoints, and not the prepared query's upstream
definition.

This commit updates prepared query clusters to account for that.
2021-08-20 17:40:33 -06:00
freddygv 1f192eb7d9 Fixup proxy config test fixtures
- The TestNodeService helper created services with the fixed name "web",
and now that name is overridable.

- The discovery chain snapshot didn't have prepared query endpoints so
the endpoints tests were missing data for prepared queries
2021-08-20 17:38:57 -06:00
Dhia Ayachi f766b6dff7
oss portion of ent #1069 (#10883) 2021-08-20 12:57:45 -04:00
Daniel Nephin 9df2464c7c xds: document how authorization works 2021-08-17 19:26:34 -04:00
Daniel Nephin 608b291565 acl: use authz consistently as the variable name for an acl.Authorizer
Follow up to https://github.com/hashicorp/consul/pull/10737#discussion_r682147950

Renames all variables for acl.Authorizer to use `authz`. Previously some
places used `rule` which I believe was an old name carried over from the
legacy ACL system.

A couple places also used authorizer.

This commit also removes another couple of authorizer nil checks that
are no longer necessary.
2021-08-17 12:14:10 -04:00
Giulio Micheloni 0bf124502e grpc Server: turn panic into error through middleware 2021-08-07 13:21:12 +01:00
Daniel Nephin 9b41e7287f acl: use acl.ManangeAll when ACLs are disabled
Instead of returning nil and checking for nilness

Removes a bunch of nil checks, and fixes one test failures.
2021-07-30 12:58:24 -04:00
R.B. Boyer 62ac98b564
agent/structs: add a bunch more EnterpriseMeta helper functions to help with partitioning (#10669) 2021-07-22 13:20:45 -05:00
Freddy a942a2e025
Merge pull request #10621 from hashicorp/vuln/validate-sans 2021-07-15 09:43:55 -06:00
R.B. Boyer e018d8a10b
xds: ensure single L7 deny intention with default deny policy does not result in allow action (CVE-2021-36213) (#10619) 2021-07-15 10:09:00 -05:00
freddygv b6b42c34dc Add TODOs about partition handling 2021-07-14 22:21:55 -06:00
freddygv 3d4fa44c22 Update golden files 2021-07-14 22:21:55 -06:00
freddygv a7de87e95b Validate SANs for passthrough clusters and failovers 2021-07-14 22:21:55 -06:00
freddygv a6f7d806f6 Update golden files to account for SAN validation 2021-07-14 22:21:55 -06:00
freddygv 3f11449363 Validate Subject Alternative Name for upstreams
These changes ensure that the identity of services dialed is
cryptographically verified.

For all upstreams we validate against SPIFFE IDs in the format used by
Consul's service mesh:

spiffe://<trust-domain>/ns/<namespace>/dc/<datacenter>/svc/<service>
2021-07-14 22:20:27 -06:00
Daniel Nephin 9c6458c6c2 rename GRPC->XDS where appropriate 2021-07-09 12:17:45 -04:00
jkirschner-hashicorp 31bbab8ae7
Merge pull request #10560 from jkirschner-hashicorp/change-sane-to-reasonable
Replace use of 'sane' where appropriate
2021-07-06 11:46:04 -04:00
Jared Kirschner 4c3b1b8b7b Replace use of 'sane' where appropriate
HashiCorp voice, style, and language guidelines recommend avoiding ableist
language unless its reference to ability is accurate in a particular use.
2021-07-02 12:18:46 -04:00
Dhia Ayachi b57cf27e8f
Format certificates properly (rfc7468) with a trailing new line (#10411)
* trim carriage return from certificates when inserting rootCA in the inMemDB

* format rootCA properly when returning the CA on the connect CA endpoint

* Fix linter warnings

* Fix providers to trim certs before returning it

* trim newlines on write when possible

* add changelog

* make sure all provider return a trailing newline after the root and intermediate certs

* Fix endpoint to return trailing new line

* Fix failing test with vault provider

* make test more robust

* make sure all provider return a trailing newline after the leaf certs

* Check for suffix before removing newline and use function

* Add comment to consul provider

* Update change log

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

* fix typo

* simplify code callflow

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

* extract requireNewLine as shared func

* remove dependency to testify in testing file

* remove extra newline in vault provider

* Add cert newline fix to envoy xds

* remove new line from mock provider

* Remove adding a new line from provider and fix it when the cert is read

* Add a comment to explain the fix

* Add missing for leaf certs

* fix missing new line

* fix missing new line in leaf certs

* remove extra new line in test

* updage changelog

Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>

* fix in vault provider and when reading cache (RPC call)

* fix AWS provider

* fix failing test in the provider

* remove comments and empty lines

* add check for empty cert in test

* fix linter warnings

* add new line for leaf and private key

* use string concat instead of Sprintf

* fix new lines for leaf signing

* preallocate slice and remove append

* Add new line to `SignIntermediate` and `CrossSignCA`

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>
Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>
2021-06-30 20:48:29 -04:00
R.B. Boyer 38d4b75ab3
xds: fix flaky protocol tests (#10410) 2021-06-16 11:57:43 -05:00
Freddy 3127dac0fb
Merge pull request #10404 from hashicorp/ingress-stats 2021-06-15 14:28:07 -06:00
R.B. Boyer 1645b6aafe
xds: adding more delta protocol tests (#10398)
Fixes #10125
2021-06-15 15:21:07 -05:00
freddygv bdb3af918c Regen golden files 2021-06-15 14:18:25 -06:00
Freddy 0d05cbb105
Update agent/xds/listeners.go
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>
2021-06-15 14:09:26 -06:00
freddygv e07542211a Remove unused param 2021-06-15 11:19:45 -06:00
freddygv 450ac44126 Update ingress gateway stats labeling
In the absence of stats_tags to handle this pattern, when we pass
"ingress_upstream.$port" as the stat_prefix, Envoy splits up that prefix
and makes the port a part of the metric name.

For example:
- stat_prefix: ingress_upstream.8080

This leads to metric names like envoy_http_8080_no_route. Changing the
stat_prefix to ingress_upstream_80880 yields the expected metric names
such as envoy_http_no_route.

Note that we don't encode the destination's name/ns/dc in this
stat_prefix because for HTTP services ingress gateways use a single
filter chain. Only cluster metrics are available on a per-upstream
basis.
2021-06-15 08:52:18 -06:00
freddygv 5d7188e290 Update terminating gateway stats labeling
This change makes it so that the stat prefix for terminating gateways
matches that of connect proxies. By using the structure of
"upstream.svc.ns.dc" we can extract labels for the destination service,
namespace, and datacenter.
2021-06-15 08:52:18 -06:00
R.B. Boyer 8d5f81b460
xds: ensure that dependent xDS resources are reconfigured during primary type warming (#10381)
Updates to a cluster will clear the associated endpoints, and updates to
a listener will clear the associated routes. Update the incremental xDS
logic to account for this implicit cleanup so that we can finish warming
the clusters and listeners.

Fixes #10379
2021-06-14 17:20:27 -05:00
Freddy f399fd2add
Rename CatalogDestinationsOnly (#10397)
CatalogDestinationsOnly is a passthrough that would enable dialing
addresses outside of Consul's catalog. However, when this flag is set to
true only _connect_ endpoints for services can be dialed.

This flag is being renamed to signal that non-Connect endpoints can't be
dialed by transparent proxies when the value is set to true.
2021-06-14 14:15:09 -06:00
Freddy 61ae2995b7
Add flag for transparent proxies to dial individual instances (#10329) 2021-06-09 14:34:17 -06:00
Freddy 62facc1a04
Revert "Avoid adding original_dst filter when not needed" (#10365) 2021-06-08 13:18:41 -06:00
Freddy c61e2bbda7
Ensure passthrough clusters can be created (#10301) 2021-05-26 15:05:14 -06:00
Freddy 7cfd7e9ec1
Avoid adding original_dst filter when not needed (#10302) 2021-05-26 15:04:45 -06:00
R.B. Boyer 7c9763d027
xds: emit a labeled gauge of connected xDS streams by version (#10243)
Fixes #10099
2021-05-14 13:59:13 -05:00
R.B. Boyer 05b52a3d63
connect: update supported envoy versions to 1.18.3, 1.17.3, 1.16.4, and 1.15.5 (#10231) 2021-05-12 14:06:06 -05:00
Daniel Nephin 55f620d636
Merge pull request #10155 from hashicorp/dnephin/config-entry-remove-fields
config-entry: remove Kind and Name field from Mesh config entry
2021-05-04 17:27:56 -04:00
Mark Anderson b9fc9ddc01 Add simple test for downstream sockets
Signed-off-by: Mark Anderson <manderson@hashicorp.com>
2021-05-04 12:41:43 -07:00
Mark Anderson 10963d0cbd Add support for downstreams
Enhance config by adding SocketPath and LocalSocketPath config values

Supports syntax of the form:
```
services {
  name = "sock_forwarder"
  id = "sock_forwarder.1"
  socket_path = "/tmp/downstream_3.sock"
  connect {
    sidecar_service {
      proxy {
	local_service_socket_path = "/tmp/downstream.sock"
      }
    }
  }
}
```

Signed-off-by: Mark Anderson <manderson@hashicorp.com>
2021-05-04 12:41:43 -07:00
Mark Anderson c3510e6d47 Add tests for xds/listeners
Signed-off-by: Mark Anderson <manderson@hashicorp.com>
2021-05-04 12:41:43 -07:00
Mark Anderson 5f04b6abe2 Convert mode to string representation
Signed-off-by: Mark Anderson <manderson@hashicorp.com>
2021-05-04 12:41:43 -07:00
Mark Anderson 626b27a874 Continue working through proxy and agent
Rework/listeners, rename makeListener

Refactor, tests pass

Signed-off-by: Mark Anderson <manderson@hashicorp.com>
2021-05-04 12:41:43 -07:00
Freddy ec38cf3206
Fixup discovery chain handling in transparent mode (#10168)
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

Previously we would associate the address of a discovery chain target
with the discovery chain's filter chain. This was broken for a few reasons:

- If the upstream is a virtual service, the client proxy has no way of
dialing it because virtual services are not targets of their discovery
chains. The targets are distinct services. This is addressed by watching
the endpoints of all upstream services, not just their discovery chain
targets.

- If multiple discovery chains resolve to the same target, that would
lead to multiple filter chains attempting to match on the target's
virtual IP. This is addressed by only matching on the upstream's virtual
IP.

NOTE: this implementation requires an intention to the redirecting
virtual service and not just to the final destination. This is how
we can know that the virtual service is an upstream to watch.

A later PR will look into traversing discovery chains when computing
upstreams so that intentions are only required to the discovery chain
targets.
2021-05-04 08:45:19 -06:00
Freddy 5427a1465c
Only consider virtual IPs for transparent proxies (#10162)
Initially we were loading every potential upstream address into Envoy
and then routing traffic to the logical upstream service. The downside
of this behavior is that traffic meant to go to a specific instance
would be load balanced across ALL instances.

Traffic to specific instance IPs should be forwarded to the original
destination and if it's a destination in the mesh then we should ensure
the appropriate certificates are used.

This PR makes transparent proxying a Kubernetes-only feature for now
since support for other environments requires generating virtual IPs,
and Consul does not do that at the moment.
2021-05-03 14:15:22 -06:00
Daniel Nephin bf4c289804 config-entry: remove Kind and Name field from Mesh config entry
No config entry needs a Kind field. It is only used to determine the Go type to
target. As we introduce new config entries (like this one) we can remove the kind field
and have the GetKind method return the single supported value.

In this case (similar to proxy-defaults) the Name field is also unnecessary. We always
use the same value. So we can omit the name field entirely.
2021-04-29 17:11:21 -04:00
R.B. Boyer 97e57aedfb
connect: update supported envoy versions to 1.18.2, 1.17.2, 1.16.3, and 1.15.4 (#10101)
The only thing that needed fixing up pertained to this section of the 1.18.x release notes:

> grpc_stats: the default value for stats_for_all_methods is switched from true to false, in order to avoid possible memory exhaustion due to an untrusted downstream sending a large number of unique method names. The previous default value was deprecated in version 1.14.0. This only changes the behavior when the value is not set. The previous behavior can be used by setting the value to true. This behavior change by be overridden by setting runtime feature envoy.deprecated_features.grpc_stats_filter_enable_stats_for_all_methods_by_default.

For now to maintain status-quo I'm explicitly setting `stats_for_all_methods=true` in all versions to avoid relying upon the default.

Additionally the naming of the emitted metrics for these gRPC requests changed slightly so the integration test assertions for `case-grpc` needed adjusting.
2021-04-29 15:22:03 -05:00
R.B. Boyer 06cd0aaa8d
xds: ensure that all envoyproxy/go-control-plane protobuf symbols are linked into the final binary (#10131)
This ensures that if someone does include some extension Consul does not currently make use of, that extension is actually usable. Without linking these envoy protobufs into the main binary it can't round trip the escape hatches to send them down to envoy.

Whenenver the go-control-plane library is upgraded next we just have to re-run 'make envoy-library'.
2021-04-29 14:58:26 -05:00
R.B. Boyer 91bee6246f
Support Incremental xDS mode (#9855)
This adds support for the Incremental xDS protocol when using xDS v3. This is best reviewed commit-by-commit and will not be squashed when merged.

Union of all commit messages follows to give an overarching summary:

xds: exclusively support incremental xDS when using xDS v3

Attempts to use SoTW via v3 will fail, much like attempts to use incremental via v2 will fail.
Work around a strange older envoy behavior involving empty CDS responses over incremental xDS.
xds: various cleanups and refactors that don't strictly concern the addition of incremental xDS support

Dissolve the connectionInfo struct in favor of per-connection ResourceGenerators instead.
Do a better job of ensuring the xds code uses a well configured logger that accurately describes the connected client.
xds: pull out checkStreamACLs method in advance of a later commit

xds: rewrite SoTW xDS protocol tests to use protobufs rather than hand-rolled json strings

In the test we very lightly reuse some of the more boring protobuf construction helper code that is also technically under test. The important thing of the protocol tests is testing the protocol. The actual inputs and outputs are largely already handled by the xds golden output tests now so these protocol tests don't have to do double-duty.

This also updates the SoTW protocol test to exclusively use xDS v2 which is the only variant of SoTW that will be supported in Consul 1.10.

xds: default xds.Server.AuthCheckFrequency at use-time instead of construction-time
2021-04-29 13:54:05 -05:00
Freddy 401f3010e0
Rename "cluster" config entry to "mesh" (#10127)
This config entry is being renamed primarily because in k8s the name
cluster could be confusing given that the config entry applies across
federated datacenters.

Additionally, this config entry will only apply to Consul as a service
mesh, so the more generic "cluster" name is not needed.
2021-04-28 16:13:29 -06:00
Freddy 6d15569062
Split Upstream.Identifier() so non-empty namespace is always prepended in ent (#10031) 2021-04-15 13:54:40 -06:00
R.B. Boyer 0470d9ec25 fix broken golden tests 2021-04-14 11:36:47 -05:00
Freddy 57b998e027
Merge pull request #9987 from hashicorp/remove-kube-dns-hack 2021-04-14 10:00:53 -06:00
freddygv 911d7dcaa8 Remove todo that was todone 2021-04-13 10:19:59 -06:00
freddygv 84a5f91a08 Avoid nil panic when cluster config doesn't exist 2021-04-13 10:17:11 -06:00
freddygv eeccba945d Replace TransparentProxy bool with ProxyMode
This PR replaces the original boolean used to configure transparent
proxy mode. It was replaced with a string mode that can be set to:

- "": Empty string is the default for when the setting should be
defaulted from other configuration like config entries.
- "direct": Direct mode is how applications originally opted into the
mesh. Proxy listeners need to be dialed directly.
- "transparent": Transparent mode enables configuring Envoy as a
transparent proxy. Traffic must be captured and redirected to the
inbound and outbound listeners.

This PR also adds a struct for transparent proxy specific configuration.
Initially this is not stored as a pointer. Will revisit that decision
before GA.
2021-04-12 09:35:14 -06:00
Iryna Shustava ff2e70f4ce
cli: Add new consul connect redirect-traffic command for applying traffic redirection rules when Transparent Proxy is enabled. (#9910)
* Add new consul connect redirect-traffic command for applying traffic redirection rules when Transparent Proxy is enabled.
* Add new iptables package for applying traffic redirection rules with iptables.
2021-04-09 11:48:10 -07:00
Freddy 38819dac1f
Merge pull request #9042 from lawliet89/tg-rewrite 2021-04-08 11:49:23 -06:00
freddygv b97d3422a7 Stable sort cidr ranges to match on 2021-04-08 11:27:57 -06:00
freddygv 69822fa5ae Remove kube-dns resolution since clusterip will be a tagged addr 2021-04-07 14:15:21 -06:00
R.B. Boyer 82245585c6
connect: add toggle to globally disable wildcard outbound network access when transparent proxy is enabled (#9973)
This adds a new config entry kind "cluster" with a single special name "cluster" where this can be controlled.
2021-04-06 13:19:59 -05:00
Yong Wen Chua 85aa7f2785
Merge branch 'master' of github.com:hashicorp/consul into tg-rewrite 2021-04-06 17:05:26 +08:00
freddygv c4ff0e6eca Uncomment listener tests 2021-03-17 21:37:12 -06:00
freddygv 6c43195e2a Merge master and fix upstream config protocol defaulting 2021-03-17 21:13:40 -06:00
freddygv ca2a62d807 Rename hasChains for clarity 2021-03-17 16:42:29 -06:00
freddygv 3c7e5c3308 PR comments 2021-03-17 16:18:56 -06:00
freddygv 4c58711594 Upstreams loop is only for prepared queries and they are not CentrallyConfigured 2021-03-17 15:32:52 -06:00
freddygv 9256d09255 Handle prepared queries in Upstreams loop and escape hatches in disco chain loop 2021-03-17 15:17:43 -06:00
freddygv 5b59780431 Update xds for transparent proxy 2021-03-17 13:40:49 -06:00
freddygv a3184e6cd7 Refactor makePublicListener
By accepting a name the function can be used for other inbound listeners,
like the one for TransparentProxy.
2021-03-16 19:22:26 -06:00
freddygv d90240d367 Restore old Envoy prefix on escape hatches
This is done because after removing ID and NodeName from
ServiceConfigRequest we will no longer know whether a request coming in
is for a Consul client earlier than v1.10.
2021-03-15 14:12:57 -06:00
freddygv c30157d2f2 Turn Limits and PassiveHealthChecks into pointers 2021-03-11 11:04:40 -07:00
freddygv 68148a1dae finish moving UpstreamConfig and related fields to structs pkg 2021-03-10 21:04:13 -07:00
freddygv 4bbd495b54 Create new types for service-defaults upstream cfg 2021-03-08 22:10:27 -07:00
R.B. Boyer 503041f216
xds: default to speaking xDS v3, but allow for v2 to be spoken upon request (#9658)
- Also add support for envoy 1.17.0
2021-02-26 16:23:15 -06:00
R.B. Boyer 4336d522c1
test: omit envoy golden test files that differ from the latest version (#9807)
Since we currently do no version switching this removes 75% of the PR
noise.

To generate all *.golden files were removed and then I ran:

    go test ./agent/xds -update
2021-02-24 14:04:31 -06:00
Yong Wen Chua 677f798443
Update test fixtures 2021-02-24 16:24:32 +08:00
Yong Wen Chua e1820517aa
Auto Rewrite Host Headers for Terminating Gateways
Tries to partially address
https://github.com/hashicorp/consul/issues/8707
2021-02-24 16:23:39 +08:00
R.B. Boyer cdc5e99184
xds: remove deprecated usages of xDS (#9602)
Note that this does NOT upgrade to xDS v3. That will come in a future PR.

Additionally:

- Ignored staticcheck warnings about how github.com/golang/protobuf is deprecated.
- Shuffled some agent/xds imports in advance of a later xDS v3 upgrade.
- Remove support for envoy 1.13.x but don't add in 1.17.x yet. We have to wait until the xDS v3 support is added in a follow-up PR.

Fixes #8425
2021-02-22 15:00:15 -06:00
R.B. Boyer e87d2bb24f
xds: only try to create an ipv6 expose checks listener if ipv6 is supported by the kernel (#9765)
Fixes #9311

This only fails if the kernel has ipv6 hard-disabled. It is not sufficient to merely not provide an ipv6 address for a network interface.
2021-02-19 14:38:43 -06:00
R.B. Boyer 194fb0d144
connect: update supported envoy point releases to 1.16.2, 1.15.3, 1.14.6, 1.13.7 (#9737) 2021-02-10 13:11:15 -06:00
Freddy 5a50b26767
Avoid potential proxycfg/xDS deadlock using non-blocking send 2021-02-08 16:14:06 -07:00
R.B. Boyer 77424e179a
xds: prevent LDS flaps in mesh gateways due to unstable datacenter lists (#9651)
Also fix a similar issue in Terminating Gateways that was masked by an overzealous test.
2021-02-08 10:19:57 -06:00
R.B. Boyer 05d767b8d6
xds: deduplicate mesh gateway listeners in a stable way (#9650)
In a situation where the mesh gateway is configured to bind to multiple
network interfaces, we use a feature called 'tagged addresses'.
Sometimes an address is duplicated across multiple tags such as 'lan'
and 'lan_ipv4'.

There is code to deduplicate these things when creating envoy listeners,
but that code doesn't ensure that the same tag wins every time. If the
winning tag flaps between xDS discovery requests it will cause the
listener to be drained and replaced.
2021-02-05 16:28:07 -06:00
freddygv 8de6b2590c Make xDS labeling consistent with proxycfg 2021-02-05 15:15:52 -07:00
freddygv a0be7dcc1d Add trace logs to proxycfg state runner and xds srv 2021-02-02 12:26:38 -07:00
Chris Boulton 448212060a
connect: add local_request_timeout_ms to configure local_app http timeouts (#9554) 2021-01-25 13:50:00 -06:00
Daniel Nephin f6543b1651 xds: remove Server.Initialize
Requiring a call to initialize to set a single field is not really substantially different
from having to set that field to a value.
2021-01-07 18:13:48 -05:00
Daniel Nephin bbf1a116f6 xds: Fix data race
TestEnvoy.Close used e.stream.recvCh == nil to indicate the channel had already
been closed, so that TestEnvoy.Close can be called multiple times. The recvCh
was not protected by a lock, so setting it to nil caused a data race with any
goroutine trying to read from the channel.

Instead set the stream to nil. The stream is guarded by a lock, so it does not race.

This change allows us to test the agent/xds package using -race.
2021-01-07 18:13:48 -05:00
Daniel Nephin de226f26e4 xds: Pass in logger
small cleanup in tests
2021-01-07 18:13:48 -05:00
Daniel Nephin ef0999547a testing: skip slow tests with -short
Add a skip condition to all tests slower than 100ms.

This change was made using `gotestsum tool slowest` with data from the
last 3 CI runs of master.
See https://github.com/gotestyourself/gotestsum#finding-and-skipping-slow-tests

With this change:

```
$ time go test -count=1 -short ./agent
ok      github.com/hashicorp/consul/agent       0.743s

real    0m4.791s

$ time go test -count=1 -short ./agent/consul
ok      github.com/hashicorp/consul/agent/consul        4.229s

real    0m8.769s
```
2020-12-07 13:42:55 -05:00
Freddy 2763833d32
Add DC and NS support for Envoy metrics (#9207)
This PR updates the tags that we generate for Envoy stats.

Several of these come with breaking changes, since we can't keep two stats prefixes for a filter.
2020-11-16 16:37:19 -07:00
R.B. Boyer 9b37ea7dcb
Revert "Add namespace support for metrics (OSS) (#9117)" (#9124)
This reverts commit 06b3b017d326853dbb53bc0ec08ce371265c5ce9.
2020-11-06 10:24:32 -06:00
Freddy 874efe705f
Add namespace support for metrics (OSS) (#9117) 2020-11-05 18:24:29 -07:00
R.B. Boyer 2183842f0e
connect: add support for envoy 1.16.0, drop support for 1.12.x, and bump point releases as well (#8944)
Supported versions will be: "1.16.0", "1.15.2", "1.14.5", "1.13.6"
2020-10-22 13:46:19 -05:00
R.B. Boyer 35c4efd220
connect: support defining intentions using layer 7 criteria (#8839)
Extend Consul’s intentions model to allow for request-based access control enforcement for HTTP-like protocols in addition to the existing connection-based enforcement for unspecified protocols (e.g. tcp).
2020-10-06 17:09:13 -05:00
R.B. Boyer d6dce2332a
connect: intentions are now managed as a new config entry kind "service-intentions" (#8834)
- Upgrade the ConfigEntry.ListAll RPC to be kind-aware so that older
copies of consul will not see new config entries it doesn't understand
replicate down.

- Add shim conversion code so that the old API/CLI method of interacting
with intentions will continue to work so long as none of these are
edited via config entry endpoints. Almost all of the read-only APIs will
continue to function indefinitely.

- Add new APIs that operate on individual intentions without IDs so that
the UI doesn't need to implement CAS operations.

- Add a new serf feature flag indicating support for
intentions-as-config-entries.

- The old line-item intentions way of interacting with the state store
will transparently flip between the legacy memdb table and the config
entry representations so that readers will never see a hiccup during
migration where the results are incomplete. It uses a piece of system
metadata to control the flip.

- The primary datacenter will begin migrating intentions into config
entries on startup once all servers in the datacenter are on a version
of Consul with the intentions-as-config-entries feature flag. When it is
complete the old state store representations will be cleared. We also
record a piece of system metadata indicating this has occurred. We use
this metadata to skip ALL of this code the next time the leader starts
up.

- The secondary datacenters continue to run the old intentions
replicator until all servers in the secondary DC and primary DC support
intentions-as-config-entries (via serf flag). Once this condition it met
the old intentions replicator ceases.

- The secondary datacenters replicate the new config entries as they are
migrated in the primary. When they detect that the primary has zeroed
it's old state store table it waits until all config entries up to that
point are replicated and then zeroes its own copy of the old state store
table. We also record a piece of system metadata indicating this has
occurred. We use this metadata to skip ALL of this code the next time
the leader starts up.
2020-10-06 13:24:05 -05:00
freddygv 60cb306524 Add session flag to cookie config 2020-09-11 18:34:03 -06:00
freddygv ae8c609f10 PR comments 2020-09-11 10:49:26 -06:00
freddygv 5871b667a5 Revert EnvoyConfig nesting 2020-09-11 09:21:43 -06:00