Commit Graph

2069 Commits

Author SHA1 Message Date
Seth Hoenig 7ea75263ea
grpc: use default resolver scheme for grpc dialing (#7617)
Currently checks of type gRPC will emit log messages such as,

    2020/02/12 13:48:22 [INFO] parsed scheme: ""
    2020/02/12 13:48:22 [INFO] scheme "" not registered, fallback to default scheme

Without adding full support for using custom gRPC schemes (maybe that's
right long-term path) we can just supply the default scheme as provided
by the grpc library.

Fixes https://github.com/hashicorp/consul/issues/7274
and https://github.com/hashicorp/nomad/issues/7415
2020-05-20 22:26:26 +02:00
Daniel Nephin e1e1c13b35 state: use an error to indicate compare failed
Errors are values. We can use the error value to identify the 'comparison failed' case which makes the function easier to use and should make it harder to miss handle the error case
2020-05-20 12:43:33 -04:00
Pierre Souchay eda40d8824
tests: added unit test to ensure watches are not re-triggered on consul reload (#7449)
This ensures no regression about https://github.com/hashicorp/consul/issues/7318
And ensure that https://github.com/hashicorp/consul/issues/7446 cannot happen anymore
2020-05-20 12:38:29 +02:00
Pierre Souchay 3b548f0d77
Allow to restrict servers that can join a given Serf Consul cluster. (#7628)
Based on work done in https://github.com/hashicorp/memberlist/pull/196
this allows to restrict the IP ranges that can join a given Serf cluster
and be a member of the cluster.

Restrictions on IPs can be done separatly using 2 new differents flags
and config options to restrict IPs for LAN and WAN Serf.
2020-05-20 11:31:19 +02:00
Daniel Nephin 6e3a7b0aa8 consul/state: refactor tnxService to avoid missed cases
Handling errors at the end of a log switch/case block is somewhat
brittle. This block included a couple cases where errors were ignored,
but it was not obvious the way it was written.

This change moves all error handling into each case block. There is
still potentially one case where err is ignored, which will be handled
in a follow up.
2020-05-19 16:50:14 -04:00
Daniel Nephin 3e76ebbd49 Remove unused var
The usage was removed in 8e22d80e3550592a32144f2d1c84ebf6becb957b,
however it seems there may be a bug here because the cluster name
is not updated when the target changes.
2020-05-19 16:50:14 -04:00
Daniel Nephin ff99f6466f Handle error from template.Execute
Refactored the function to make the problem more obvious, by using a
guard.
2020-05-19 16:50:14 -04:00
Daniel Nephin 545bd766e7 Fix a number of problems found by staticcheck
Some of these problems are minor (unused vars), but others are real bugs (ignored errors).

Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com>
2020-05-19 16:50:14 -04:00
Daniel Nephin aa8009ee45 Remove unused var
The usage of this var was removed in b92f895c233bf8a99ae35361117a90416fea29b5.

Found by using staticcheck
2020-05-19 16:50:14 -04:00
Chris Piraino 0995906b24 Add service id context to the proxycfg logger
This is especially useful when multiple proxies are all querying the
same Consul agent.
2020-05-18 09:08:05 -05:00
Chris Piraino ce099c9aca Do not return an error if requested service is not a gateway
This commit converts the previous error into just a Warn-level log
message. By returning an error when the requested service was not a
gateway, we did not appropriately update envoy because the cache Fetch
returned an error and thus did not propagate the update through proxycfg
and xds packages.
2020-05-18 09:08:04 -05:00
Aleksandr Zagaevskiy 75f0607d3b
Preserve ModifyIndex for unchanged entry in KVS TXN (#7832) 2020-05-14 13:25:04 -06:00
Pierre Souchay 9611a266a2
tests: fix unstable test `TestAgentAntiEntropy_Checks`. (#7594)
Example of failure: https://circleci.com/gh/hashicorp/consul/153932#tests/containers/2
2020-05-14 09:54:49 +02:00
Kit Patella a33791c66a http: migrate from instrumentation in s.wrap() to an s.enterpriseHandler() 2020-05-13 15:47:05 -07:00
Matt Keeler 849eedd142
Fix identity resolution on clients and in secondary dcs (#7862)
Previously this happened to be using the method on the Server/Client that was meant to allow the ACLResolver to locally resolve tokens. On Servers that had tokens (primary or secondary dc + token replication) this function would lookup the token from raft and return the ACLIdentity. On clients this was always a noop. We inadvertently used this function instead of creating a new one when we added logging accessor ids for permission denied RPC requests. 

With this commit, a new method is used for resolving the identity properly via the ACLResolver which may still resolve locally in the case of being on a server with tokens but also supports remote token resolution.
2020-05-13 13:00:08 -04:00
Chris Piraino 3eb4e4012a
Make new gateway tests compatible with enterprise (#7856) 2020-05-12 13:48:20 -05:00
Daniel Nephin 2e0f750f1a Add unconvert linter
To find unnecessary type convertions
2020-05-12 13:47:25 -04:00
Drew Bailey 7e1734b1c6 Value is already an int, remove type cast 2020-05-12 13:13:09 -04:00
Daniel Nephin 0985eff717
Merge pull request #7689 from hashicorp/dnephin/remove-deadcode-1
Remove some dead code
2020-05-12 12:33:59 -04:00
Daniel Nephin ad2979ca45
Merge pull request #7819 from hashicorp/dnephin/remove-t.Parallel-1
test: Remove t.Parallel() from agent/structs tests
2020-05-12 12:11:57 -04:00
R.B. Boyer 940e5ad160
acl: add auth method for JWTs (#7846) 2020-05-11 20:59:29 -05:00
Kit Patella 24175e2925
Merge pull request #7843 from hashicorp/oss-sync/auditing-config
agent/config: Fix tests & include Audit struct as a pointer on Config
2020-05-11 14:23:44 -07:00
Kit Patella a4ee78440b agent/config: include Audit struct as a pointer on Config, fix tests 2020-05-11 14:13:05 -07:00
Kit Patella a45b67b40f
Merge pull request #7841 from hashicorp/oss-sync/auditing-config
OSS sync - Auditing config
2020-05-11 13:44:38 -07:00
Kit Patella f82a97f8c8 agent/config: add auditing config to OSS and add to enterpriseConfigMap exclusions 2020-05-11 13:27:35 -07:00
Chris Piraino 1173b31949
Return early from updateGatewayServices if nothing to update (#7838)
* Return early from updateGatewayServices if nothing to update

Previously, we returned an empty slice of gatewayServices, which caused
us to accidentally delete everything in the memdb table

* PR comment and better formatting
2020-05-11 14:46:48 -05:00
Chris Piraino 1a7c99cf31
Fix TestInternal_GatewayServiceDump_Ingress (#7840)
Protocol was added as a field on GatewayServices after
GatewayServiceDump PR branch was created.
2020-05-11 14:46:31 -05:00
R.B. Boyer c54211ad52
cli: ensure 'acl auth-method update' doesn't deep merge the Config field (#7839) 2020-05-11 14:21:17 -05:00
Chris Piraino 107c7a9ca7 PR comment and better formatting 2020-05-11 14:04:59 -05:00
Chris Piraino 9f924400e0 Return early from updateGatewayServices if nothing to update
Previously, we returned an empty slice of gatewayServices, which caused
us to accidentally delete everything in the memdb table
2020-05-11 12:38:04 -05:00
Freddy ebbb234ecb
Gateway Services Nodes UI Endpoint (#7685)
The endpoint supports queries for both Ingress Gateways and Terminating Gateways. Used to display a gateway's linked services in the UI.
2020-05-11 11:35:17 -06:00
Kyle Havlovitz 28b4819882
Merge pull request #7759 from hashicorp/ingress/tls-hosts
Add TLS option for Ingress Gateway listeners
2020-05-11 09:18:43 -07:00
Kyle Havlovitz dc1112eaf8 Disallow the blanket wildcard prefix from being used as custom host 2020-05-08 20:24:18 -07:00
Chris Piraino 3931015a90 Remove development log line 2020-05-08 20:24:18 -07:00
Chris Piraino a500262a77 Compute all valid DNSSANs for ingress gateways
For DNSSANs we take into account the following and compute the
appropriate wildcard values:
- source datacenter
- namespaces
- alt domains
2020-05-08 20:23:17 -07:00
Daniel Nephin ce3aeb85f5 Add outlier_detection check to integration test
Fix decoding of time.Duration types.
2020-05-08 14:56:57 -04:00
Daniel Nephin 987875524a xds: Add passive health check config for upstreams 2020-05-08 14:56:57 -04:00
Chris Piraino a635e23f86
Restoring config entries updates the gateway-services table (#7811)
- Adds a new validateConfigEntryEnterprise function
- Also fixes some state store tests that were failing in enterprise
2020-05-08 13:24:33 -05:00
Daniel Nephin c2499418ed test: Remove t.Parallel() from agent/structs tests
go test will only run tests in parallel within a single package. In this case the package test run time is exactly the same with or without t.Parallel() (~0.7s).

In generally we should avoid t.Parallel() as it causes a number of problems with `go test` not reporting failure messages correctly. I encountered one of these problems, which is what prompted this change.  Since `t.Parallel` is not providing any benefit in this package, this commit removes it.

The change was automated with:

    git grep -l 't.Parallel' | xargs sed -i -e '/t.Parallel/d'
2020-05-08 14:06:10 -04:00
Freddy a37d7a42c9
Fix up enterprise compatibility for gateways (#7813) 2020-05-08 09:44:34 -06:00
Jono Sosulska 44011c81f2
Fix spelling of deregister (#7804) 2020-05-08 10:03:45 -04:00
Chris Piraino 2d657c3c0f
Allow ingress gateways to send empty clusters, routes, and listeners (#7795)
This is useful when updating an config entry with no services, and the
expected behavior is that envoy closes all listeners and clusters.

We also allow empty routes because ingress gateways name route
configurations based on the port of the listener, so it is important we
remove any stale routes. Then, if a new listener with an old port is
added, we will not have to deal with stale routes hanging around routing
to the wrong place.

Endpoints are associated with clusters, and thus by deleting the
clusters we don't have to care about sending empty endpoint responses.
2020-05-07 16:19:25 -05:00
Chris Piraino 964e55e45e Cleanup proxycfg for TLS
- Use correct enterprise metadata for finding config entry
- nil out cancel functions on config snapshot copy
- Look at HostsSet when checking validity
2020-05-07 10:22:57 -05:00
Chris Piraino ad8a0544f2
Require individual services in ingress entry to match protocols (#7774)
We require any non-wildcard services to match the protocol defined in
the listener on write, so that we can maintain a consistent experience
through ingress gateways. This also helps guard against accidental
misconfiguration by a user.

- Update tests that require an updated protocol for ingress gateways
2020-05-06 16:09:24 -05:00
Freddy a749f46316
Remove timeout and call to Fatal from goroutine (#7797) 2020-05-06 14:33:17 -06:00
Chris Piraino bebf1d5df9 Add TLS field to ingress API structs
- Adds test in api and command/config/write packages
2020-05-06 15:12:02 -05:00
Chris Piraino 2a10984efb Add test for adding DNSSAN for ConnectCALeaf cache type 2020-05-06 15:12:02 -05:00
Chris Piraino 91586b9228 Validate hosts input in ingress gateway config entry
We can only allow host names that are valid domain names because we put
these hosts into a DNSSAN. In addition, we validate that the wildcard
specifier '*' is only present as the leftmost label to allow for a
wildcard DNSSAN and associated wildcard Host routing in the ingress
gateway proxy.
2020-05-06 15:12:02 -05:00
Kyle Havlovitz bd6bb3bf2d Add TLS option and DNS SAN support to ingress config
xds: Only set TLS context for ingress listener when requested
2020-05-06 15:12:02 -05:00
Chris Piraino ac115e39b2 A proxy-default config entry only exists in the default namespace 2020-05-06 15:06:14 -05:00
Chris Piraino ff501ffb40 Correctly set a namespace label in the required domain for xds routes
If an upstream is not in the default namespace, we expect DNS requests
to be served over "<service-name>.ingress.<namespace>.*"
2020-05-06 15:06:14 -05:00
Chris Piraino 9a130f2ccc Remove outdated comment 2020-05-06 15:06:14 -05:00
Chris Piraino 21e0ed8f8d Better document wildcard specifier interactions 2020-05-06 15:06:14 -05:00
Chris Piraino d8ac79041d Re-add comment on connect-proxy virtual hosts 2020-05-06 15:06:14 -05:00
Kyle Havlovitz 26533dcb09 Make sure IngressHosts isn't parsed during JSON decode 2020-05-06 15:06:14 -05:00
Chris Piraino c30a7bfdfe Comment why it is ok to expect upstreams slice to not be empty 2020-05-06 15:06:13 -05:00
Chris Piraino 9b9dfa8eeb xds: Use only the port number as the configured route name
This removes duplication of protocol from the stats_prefix
2020-05-06 15:06:13 -05:00
Kyle Havlovitz 04b6bd637a Filter wildcard gateway services to match listener protocol
This now requires some type of protocol setting in ingress gateway tests
to ensure the services are not filtered out.

- small refactor to add a max(x, y) function
- Use internal configEntryTxn function and add MaxUint64 to lib
2020-05-06 15:06:13 -05:00
Chris Piraino 210dda5682 Allow Hosts field to be set on an ingress config entry
- Validate that this cannot be set on a 'tcp' listener nor on a wildcard
service.
- Add Hosts field to api and test in consul config write CLI
- xds: Configure envoy with user-provided hosts from ingress gateways
2020-05-06 15:06:13 -05:00
Chris Piraino 837d2aa7d2 Remove service_subset field from ingress config entry
We decided that this was not a useful MVP feature, and just added
unnecessary complexity
2020-05-06 15:06:13 -05:00
Kyle Havlovitz e4268c8b7f Support multiple listeners referencing the same service in gateway definitions 2020-05-06 15:06:13 -05:00
Kyle Havlovitz b21cd112e5 Allow ingress gateways to route traffic based on Host header
This commit adds the necessary changes to allow an ingress gateway to
route traffic from a single defined port to multiple different upstream
services in the Consul mesh.

To do this, we now require all HTTP requests coming into the ingress
gateway to specify a Host header that matches "<service-name>.*" in
order to correctly route traffic to the correct service.

- Differentiate multiple listener's route names by port
- Adds a case in xds for allowing default discovery chains to create a
  route configuration when on an ingress gateway. This allows default
  services to easily use host header routing
- ingress-gateways have a single route config for each listener
  that utilizes domain matching to route to different services.
2020-05-06 15:06:13 -05:00
R.B. Boyer 1187d7288e
acl: oss plumbing to support auth method namespace rules in enterprise (#7794)
This includes website docs updates.
2020-05-06 13:48:04 -05:00
R.B. Boyer b6cc92020d
test: make the kube auth method test helper use freeport (#7788) 2020-05-05 16:55:21 -05:00
Hans Hasselberg e3e2b82a00 network_segments: stop advertising segment tags 2020-05-05 21:32:05 +02:00
Hans Hasselberg 854aac510f agent: refactor to use a single addrFn 2020-05-05 21:08:10 +02:00
Hans Hasselberg 0f2e189012 agent: rename local/global to src/dst 2020-05-05 21:07:34 +02:00
Chris Piraino 837bd6f558
Construct a default destination if one does not exist for service-router (#7783) 2020-05-05 10:49:50 -05:00
R.B. Boyer c9c557477b
acl: add MaxTokenTTL field to auth methods (#7779)
When set to a non zero value it will limit the ExpirationTime of all
tokens created via the auth method.
2020-05-04 17:02:57 -05:00
R.B. Boyer 265d2ea9e1
acl: add DisplayName field to auth methods (#7769)
Also add a few missing acl fields in the api.
2020-05-04 15:18:25 -05:00
Hans Hasselberg 1be90e0fa1
agent: don't let left nodes hold onto their node-id (#7747) 2020-05-04 18:39:08 +02:00
Matt Keeler 669d22933e
Merge pull request #7714 from hashicorp/oss-sync/msp-agent-token 2020-05-04 11:33:50 -04:00
Matt Keeler 64baf36b60
Update enterprise configurations to be in OSS
This will emit warnings about the configs not doing anything but still allow them to be parsed.

This also added the warnings for enterprise fields that we already had in OSS but didn’t change their enforcement behavior. For example, attempting to use a network segment will cause a hard error in OSS.
2020-05-04 10:21:05 -04:00
R.B. Boyer 3ac5a841ec
acl: refactor the authmethod.Validator interface (#7760)
This is a collection of refactors that make upcoming PRs easier to digest.

The main change is the introduction of the authmethod.Identity struct.
In the one and only current auth method (type=kubernetes) all of the
trusted identity attributes are both selectable and projectable, so they
were just passed around as a map[string]string.

When namespaces were added, this was slightly changed so that the
enterprise metadata can also come back from the login operation, so
login now returned two fields.

Now with some upcoming auth methods it won't be true that all identity
attributes will be both selectable and projectable, so rather than
update the login function to return 3 pieces of data it seemed worth it
to wrap those fields up and give them a proper name.
2020-05-01 17:35:28 -05:00
R.B. Boyer 4cd1d62e40
acl: change authmethod.Validator to take a logger (#7758) 2020-05-01 15:55:26 -05:00
R.B. Boyer 4a630135b8
test: move some test helpers over from enterprise (#7754) 2020-05-01 14:52:15 -05:00
R.B. Boyer 9faf8c42d1
sdk: extracting testutil.RequireErrorContains from various places it was duplicated (#7753) 2020-05-01 11:56:34 -05:00
Hans Hasselberg 6626cb69d6
rpc: oss changes for network area connection pooling (#7735) 2020-04-30 22:12:17 +02:00
Freddy c34ee5d339
Watch fallback channel for gateways that do not exist (#7715)
Also ensure that WatchSets in tests are reset between calls to watchFired. 
Any time a watch fires, subsequent calls to watchFired on the same WatchSet
will also return true even if there were no changes.
2020-04-29 16:52:27 -06:00
Matt Keeler fbb5c28c80
Updates to allow for using an enterprise specific token as the agents token
This is needed to allow for managed Consul instances to register themselves in the catalog with one of the managed service provider tokens.
2020-04-28 09:44:26 -04:00
Matt Keeler 901d6739ad
Some boilerplate to allow for ACL Bootstrap disabling configurability 2020-04-28 09:42:46 -04:00
Freddy f5c1e5268b
TLS Origination for Terminating Gateways (#7671) 2020-04-27 16:25:37 -06:00
freddygv 0ad47bdda5 Remove fallthrough 2020-04-27 12:00:14 -06:00
freddygv ac809ec635 Add authz filter when creating filterchain 2020-04-27 11:08:41 -06:00
freddygv 3a44359d7b Fix snapshot IsEmpty 2020-04-27 11:08:41 -06:00
freddygv e751b83a3f Clean up dead code, issue addressed by passing ws to serviceGatewayNodes 2020-04-27 11:08:41 -06:00
Freddy a54f2fd055 Update agent/proxycfg/state_test.go 2020-04-27 11:08:41 -06:00
freddygv e30d64289d PR comments 2020-04-27 11:08:41 -06:00
freddygv 75e737b0f2 Fix internal endpoint test 2020-04-27 11:08:41 -06:00
freddygv 8697ab71e4 Fix listener error handling 2020-04-27 11:08:41 -06:00
freddygv 6ecb3b7a42 Skip filter chain creation if no client cert 2020-04-27 11:08:41 -06:00
freddygv 7667567688 Avoid deleting mappings for services linked to other gateways on dereg 2020-04-27 11:08:41 -06:00
freddygv 28fe6920fe Re-fix bug in CheckConnectServiceNodes 2020-04-27 11:08:41 -06:00
freddygv 5e260b2913 Use golden files for gateway certs and fix listener test flakiness 2020-04-27 11:08:41 -06:00
freddygv b2b5942f4b Un-nest switch in gateway update handler 2020-04-27 11:08:40 -06:00
freddygv 3849b066a6 Allow terminating-gateway to setup listener before servicegroups are known 2020-04-27 11:08:40 -06:00
freddygv 929491c979 Add subset support 2020-04-27 11:08:40 -06:00
freddygv bab101107c Fix ConnectQueryBlocking test 2020-04-27 11:08:40 -06:00
freddygv 65e60d02f1 Fix bug in CheckConnectServiceNodes
Previously, if a blocking query called CheckConnectServiceNodes
before the gateway-services memdb table had any entries,
a nil watchCh would be returned when calling serviceTerminatingGatewayNodes.
This means that the blocking query would not fire if a gateway config entry
was added after the watch started.

In cases where the blocking query started on proxy registration,
the proxy could potentially never become aware of an upstream endpoint
if that upstream was going to be represented by a gateway.
2020-04-27 11:08:40 -06:00
freddygv 2e35a9bb18 Add xds cluster/listener/endpoint management 2020-04-27 11:08:40 -06:00