571324ad08
Tune streaming backoff on errors to retry a bit faster when TCP connections drop
2021-04-07 14:13:30 +01:00
f4257f91f6
Set gRPC keepalives to mirror Yamux keepalive behaviour
2021-04-07 14:09:22 +01:00
df538b9bdc
missed build tag on this file ( #9974 )
2021-04-06 13:24:11 -05:00
82245585c6
connect: add toggle to globally disable wildcard outbound network access when transparent proxy is enabled ( #9973 )
...
This adds a new config entry kind "cluster" with a single special name "cluster" where this can be controlled.
2021-04-06 13:19:59 -05:00
a711e119e7
Merge pull request #9958 from hashicorp/dnephin/state-improve-indexer-tests
...
state: support additional test cases in indexer tests
2021-04-06 11:55:24 -04:00
85aa7f2785
Merge branch 'master' of github.com:hashicorp/consul into tg-rewrite
2021-04-06 17:05:26 +08:00
af78561018
api: ensure v1/health/ingress/:service endpoint works properly when streaming is enabled ( #9967 )
...
The streaming cache type for service health has no way to handle v1/health/ingress/:service queries as there is no equivalent topic that would return the appropriate data.
Ensure that attempts to use this endpoint will use the old cache-type for now so that they return appropriate data when streaming is enabled.
2021-04-05 13:23:00 -05:00
f0590e7c18
state: support additional test cases in indexer tests
...
And add a few additional cases.
2021-03-31 14:39:33 -04:00
3cdd495600
Backport enterprise changes to prevent merge conflicts
...
Co-Authored-By: Kyle Havlovitz <kylehav@gmail.com>
2021-03-31 14:05:26 -04:00
de16417be1
Merge pull request #9949 from hashicorp/dnephin/state-index-checks
...
state: convert remaining checks table indexers to functional pattern
2021-03-31 11:53:21 -04:00
3477e70c11
Merge pull request #9948 from hashicorp/dnephin/state-index-service
...
state: convert remaining services table indexers to functions
2021-03-31 11:49:21 -04:00
3df239b036
Merge pull request #9947 from hashicorp/dnephin/state-ent-index-3
...
state: move indexer functions out of oss files
2021-03-31 11:45:26 -04:00
68c91b535b
auto-updated agent/uiserver/bindata_assetfs.go from commit ee5c3e3aa
2021-03-31 15:00:21 +00:00
e4a60a2a8d
state: use tableIndex constant
2021-03-29 18:52:20 -04:00
7cb2255838
state: use ReadTxn and WriteTxn interface
...
Instead of *txn, so that we can replace the txn implementation with others, and so
that the function is easily documented as a read or write function.
2021-03-29 18:52:16 -04:00
eded4dfc78
Merge pull request #9932 from hashicorp/dnephin/use-ent-meta-default
...
Set default enterprise meta in test case
2021-03-29 18:44:39 -04:00
d785c86db1
state: convert checks.service index to new pattern
2021-03-29 16:38:53 -04:00
f859ba6d4b
state: convert checks.status indexer
...
As part of this change the indexer will now be case insensitive by using
the lower case value. This should be safe because previously we always
had lower case strings.
This change was made out of convenience. All the other indexers use
lowercase, so we can re-use the indexFromQuery function by using
lowercase here as well.
2021-03-29 16:38:50 -04:00
9251ac881a
state: add tests for checks indexers
2021-03-29 16:38:47 -04:00
98d6dcbdf8
state: use constants for table checks
2021-03-29 16:38:43 -04:00
bcbdc9cab3
state: pass Query in from caller
...
To reduce the number of arguments
2021-03-29 15:42:30 -04:00
372d274b34
state: convert services.kind to functional indexer pattern
2021-03-29 15:42:30 -04:00
bcde8d2fad
state: add tests for services.kind indexer
2021-03-29 15:42:27 -04:00
9f9eadd569
state: convert services table service and connect indexer
...
To the new functional indexer pattern
2021-03-29 15:42:24 -04:00
11311c1fcc
state: add tests for services table service and connect indexers
2021-03-29 15:42:22 -04:00
9a3daf3100
state: use constant for tableServices
2021-03-29 15:42:18 -04:00
ec04df66bd
state: remove duplication of Query indexer
2021-03-29 14:35:11 -04:00
28866e48ad
state: remove duplication in acl tables schema
2021-03-29 14:21:27 -04:00
c6a1ca701d
state: reduce duplication in catalog table schema
2021-03-29 14:21:23 -04:00
d9dacb8388
state: share more indexer functions for config_entries
2021-03-29 14:21:20 -04:00
f303120f2d
state: remove old schema test
...
This test has been replaced by TestNewDBSchema_Indexers
2021-03-29 14:21:13 -04:00
150decff2a
state: use addNamespaceIndex again
2021-03-29 14:21:02 -04:00
5bba621bda
auto-updated agent/uiserver/bindata_assetfs.go from commit 4d13e31ae
2021-03-26 15:51:59 +00:00
f132efde98
Set default enterprise meta in test case
2021-03-25 17:40:22 -04:00
d1b05f14c7
agent: use testAgent_RegisterService_TranslateKeys()
...
When this test was refactored it looks like there was a copy+paste error
and the test case was made to call the wrong function.
2021-03-25 16:55:13 -04:00
6aedd2f7d9
agent: remove unused makeTelemetryDefaults()
2021-03-25 16:02:42 -04:00
197d9539a7
Merge pull request #9923 from hashicorp/dnephin/fix-ui-config
...
http: fix a bug that would cause runtimeConfig to be cached
2021-03-25 12:26:09 -04:00
4a3b462c28
Merge pull request #9911 from hashicorp/dnephin/state-index-acl-roles
...
state: convert ACLRoles policies index to new functional indexer pattern
2021-03-24 18:28:19 -04:00
6731cf6061
Merge pull request #9916 from hashicorp/dnephin/state-index-checks-id
...
state: convert checks.ID index to the functional indexer pattern
2021-03-24 18:23:52 -04:00
44e32271a4
http: add a test for transform changing value
2021-03-24 15:38:11 -04:00
ae4525e412
http: fix a bug that would cause runtimeConfig to be cached
...
This bug would result in the UI not having the correct settings in
Consul enterprise, which could produce many warnings in the logs.
This bug occured because the index page, which includes a map of configuration
was rendered when the HTTPHandler is first created. This PR changes the
UIServer to instead render the index page when the page is requested.
The rendering does not appear to be all that expensive, so rendering it
when requested should not cause much extra latency.
2021-03-24 14:48:18 -04:00
b7679870df
auto-updated agent/uiserver/bindata_assetfs.go from commit 232921b60
2021-03-23 13:26:56 +00:00
25b791ba47
state: add tests for checks.ID indexer
2021-03-22 18:06:43 -04:00
abbe5c3701
state: use tx.First instead of tx.FirstWatch
...
Where appropriate. After removing the helper function a bunch of these calls can
be changed to tx.First.
2021-03-22 18:06:33 -04:00
49938bc472
state: convert checks.ID index to new pattern
2021-03-22 18:06:08 -04:00
052662bcf9
introduce certopts ( #9606 )
...
* introduce cert opts
* it should be using the same signer
* lint and omit serial
2021-03-22 10:16:41 +01:00
1d3fe64bba
state: use uuid for acl-roles.policies index
...
Previously we were encoding the UUID as a string, but the index it references uses a UUID
so this index can also use an encoded UUID to save a bit of memory.
2021-03-19 19:45:37 -04:00
3c01bb1156
state: convert acl-roles.policies index to new pattern
2021-03-19 19:45:37 -04:00
474e95b9f5
state: convert acl-roles.name index to the functional indexer pattern
2021-03-19 19:45:37 -04:00
f836ed256b
state: add indexer tests for acl-roles table
2021-03-19 19:45:37 -04:00
6bc2c0e1ce
state: use constants for acl-roles table and indexes
2021-03-19 19:45:37 -04:00
d4e02024fe
state: convert acl-policies table to new pattern
2021-03-19 15:24:00 -04:00
845a10354e
state: use constants and add tests for acl-policies table
2021-03-19 15:19:57 -04:00
f6533a08f8
state: add indexer test for services.ID index
2021-03-19 14:13:14 -04:00
1d1c03d0cd
state: handle wildcard for services.ID index
...
When listing services, use the id_prefix directly if wildcards are allowed.
Error if a wildcard is used for a query that does not index the wildcard
2021-03-19 14:12:19 -04:00
bae69b2352
state: fix prefix index with the new pattern
...
Prefix queries are generally being used to match part of a partial
index. We can support these indexes by using a function that accept
different types for each subset of the index.
What I found interesting is that in the generic StringFieldIndexer the
implementation for PrefixFromArgs would remove the trailing null, but
at least in these 2 cases we actually want a null terminated string.
We simply want fewer components in the string.
2021-03-19 14:12:17 -04:00
ec50454fb3
state: move services.ID to new pattern
2021-03-19 14:11:59 -04:00
613f9fbffc
auto-updated agent/uiserver/bindata_assetfs.go from commit a7a56ca39
2021-03-19 15:19:17 +00:00
f5a52a4501
state: add tests for gateway-service table indexers
2021-03-18 12:09:42 -04:00
66632538d8
state: use constants and remove wrapping
...
for GatewayServices table
2021-03-18 12:08:59 -04:00
d77bdd26c5
state: Move UpstreamDownstream to state package
2021-03-18 12:08:59 -04:00
ca3686f4aa
state: add tests for mesh-topology table indexers
2021-03-18 12:08:57 -04:00
8a1a11814d
state: use constants for mesh-topology table operations
2021-03-18 12:08:03 -04:00
a736a9cec4
auto-updated agent/uiserver/bindata_assetfs.go from commit 980299d51
2021-03-18 15:23:26 +00:00
c7b3bd1915
auto-updated agent/uiserver/bindata_assetfs.go from commit 8dc590cf1
2021-03-18 14:41:20 +00:00
8ac9f2521b
Merge pull request #9900 from hashicorp/ent-fixes
...
Fixup enterprise tests from tproxy changes
2021-03-18 08:33:30 -06:00
28c29e6ab4
Merge pull request #9899 from hashicorp/wildcard-ixn-oss
...
Add methods to check intention has wildcard src or dst
2021-03-18 08:33:07 -06:00
b56bd690aa
Fixup enterprise tests from tproxy changes
2021-03-17 23:05:00 -06:00
1c46470a29
Add methods to check intention has wildcard src or dst
2021-03-17 22:15:48 -06:00
291d7562d1
Cancel watch on all errors
2021-03-17 21:44:14 -06:00
de7f2a1a74
Split up normalizing from defaulting values for upstream cfg
2021-03-17 21:37:55 -06:00
c4ff0e6eca
Uncomment listener tests
2021-03-17 21:37:12 -06:00
6c43195e2a
Merge master and fix upstream config protocol defaulting
2021-03-17 21:13:40 -06:00
0c8b618ca0
Temporarily silence spurious wakeup. Addressing false positive in beta.
2021-03-17 17:25:29 -06:00
60690cf5c9
Merge remote-tracking branch 'origin/master' into intention-topology-endpoint
2021-03-17 17:14:38 -06:00
63dcb7fa76
Add TransparentProxy option to proxy definitions
2021-03-17 17:01:45 -06:00
fb252e87a4
Add per-upstream configuration to service-defaults
2021-03-17 16:59:51 -06:00
15a145b9f6
Add changelog and cleanup todo for beta
2021-03-17 16:45:13 -06:00
ca2a62d807
Rename hasChains for clarity
2021-03-17 16:42:29 -06:00
3c7e5c3308
PR comments
2021-03-17 16:18:56 -06:00
4c58711594
Upstreams loop is only for prepared queries and they are not CentrallyConfigured
2021-03-17 15:32:52 -06:00
9256d09255
Handle prepared queries in Upstreams loop and escape hatches in disco chain loop
2021-03-17 15:17:43 -06:00
5b59780431
Update xds for transparent proxy
2021-03-17 13:40:49 -06:00
3c97e5a777
Update proxycfg for transparent proxy
2021-03-17 13:40:39 -06:00
d19a5830dd
Do not include consul as upstream or downstream
2021-03-17 13:40:04 -06:00
d2591312f8
state: add tests for config-entry indexers
2021-03-17 14:41:46 -04:00
1b8f8b135e
state: convert config-entries kind index to new pattern
2021-03-17 14:40:57 -04:00
bfcf463c3a
state: remove config-entries namespace index
...
Use a prefix of the ID index instead.
2021-03-17 14:40:57 -04:00
dcbb1ba5dd
state: remove unnecessary method receiver
2021-03-17 14:40:57 -04:00
b43977423f
state: convert config-entries table to new indexer pattern
...
Using functional indexes to isolate enterprise differentiation and
remove reflection.
2021-03-17 14:40:57 -04:00
98c32599e4
Merge pull request #9881 from hashicorp/dnephin/state-index-service-check-nodes
...
state: convert services.node and checks.node indexes
2021-03-17 14:12:02 -04:00
b771baa1f5
Merge pull request #9863 from hashicorp/dnephin/config-entry-kind-name
...
state: move ConfigEntryKindName
2021-03-17 14:09:39 -04:00
110da59c6c
auto-updated agent/uiserver/bindata_assetfs.go from commit 9e715842d
2021-03-17 16:03:18 +00:00
fab276cb7d
auto-updated agent/uiserver/bindata_assetfs.go from commit f9e8b26af
2021-03-17 14:45:58 +00:00
d38917b12b
auto-updated agent/uiserver/bindata_assetfs.go from commit aca797658
2021-03-17 11:27:44 +00:00
7123aaad92
auto-updated agent/uiserver/bindata_assetfs.go from commit 41471719e
2021-03-17 10:50:59 +00:00
a3184e6cd7
Refactor makePublicListener
...
By accepting a name the function can be used for other inbound listeners,
like the one for TransparentProxy.
2021-03-16 19:22:26 -06:00
94b02c3954
Add support for configuring TLS ServerName for health checks
...
Some TLS servers require SNI, but the Golang HTTP client doesn't
include it in the ClientHello when connecting to an IP address. This
change adds a new TLSServerName field to health check definitions to
optionally set it. This fixes #9473 .
2021-03-16 18:16:44 -04:00
cba952a6a5
Add cache-type for Internal.IntentionUpstreams
2021-03-16 11:06:47 -06:00
0b3930272d
state: convert services.node and checks.node indexes
...
Using NodeIdentity to share the indexes with both.
2021-03-16 13:00:31 -04:00
b79039c21c
Prefix match type vars to match use
2021-03-16 09:49:24 -06:00
fed983fe9a
Pass txn into service list queries
2021-03-16 09:33:08 -06:00
26ba0c0fc8
Pass txn into intention match queries
2021-03-16 08:03:52 -06:00
d7f3bcc8bb
Replace CertURI.Authorize() calls.
...
AuthorizeIntentionTarget is a generalized version of the old function,
and can be evaluated against sources or destinations.
2021-03-15 18:06:04 -06:00
eb6c0cbea0
Fixup typo, comments, and regression
2021-03-15 17:50:47 -06:00
9bfb0969f9
Fixup upstream test
2021-03-15 17:20:30 -06:00
940b7a98d1
Finish cleanup from ServiceConfigRequest changes
2021-03-15 16:38:01 -06:00
a67c92b961
Update service manager to pass MeshGateway with config req
2021-03-15 16:08:03 -06:00
871e1d3e31
PR comments
2021-03-15 16:02:03 -06:00
0b5dfee00a
state: use runCase pattern for large test
...
The TestServiceHealthEventsFromChanges function was over 1400 lines.
Attempting to debug test failures in test functions this large is
difficult. It requires scrolling to the line which defines the testcase
because the failure message only includes the line number of the
assertion, not the line number of the test case.
This is an excellent example of where test tables stop working well, and
start being a problem. To mitigate this problem, the runCase pattern can
be used. When one of these tests fails, a failure message will print the
line number of both the test case and the assertion. This allows a
developer to quickly jump to both of the relevant lines, signficanting
reducing the time it takes to debug test failures.
For example, one such failure could look like this:
catalog_events_test.go:1610: case: service reg, new node
catalog_events_test.go:1605: assertion failed: values are not equal
2021-03-15 17:53:16 -04:00
04fbc104cd
Pass MeshGateway config in service config request
...
ResolveServiceConfig is called by service manager before the proxy
registration is in the catalog. Therefore we should pass proxy
registration flags in the request rather than trying to fetch
them from the state store (where they may not exist yet).
2021-03-15 14:32:13 -06:00
d90240d367
Restore old Envoy prefix on escape hatches
...
This is done because after removing ID and NodeName from
ServiceConfigRequest we will no longer know whether a request coming in
is for a Consul client earlier than v1.10.
2021-03-15 14:12:57 -06:00
13cce3419a
Only lowercase the protocol when normalizing
2021-03-15 14:12:15 -06:00
f584c2d7c5
Add omitempty across the board for UpstreamConfig
2021-03-15 13:23:18 -06:00
3b2169b36d
Add RPC endpoint for intention upstreams
2021-03-15 08:50:35 -06:00
e4e14639b2
Add state store function for intention upstreams
2021-03-15 08:50:35 -06:00
4976c000b7
Refactor IntentionDecision
...
This enables it to be called for many upstreams or downstreams of a
service while only querying intentions once.
Additionally, decisions are now optionally denied due to L7 permissions
being present. This enables the function to be used to filter for
potential upstreams/downstreams of a service.
2021-03-15 08:50:35 -06:00
2a53b8293a
proxycfg: use rpcclient/health.Client instead of passing around cache name
...
This should allow us to swap out the implementation with something other
than `agent/cache` without making further code changes.
2021-03-12 11:46:04 -05:00
c33570be34
catalog_events: set the right key for connect snapshots
2021-03-12 11:35:43 -05:00
410b1261c2
proxycfg: Use streaming in connect state
2021-03-12 11:35:42 -05:00
e2215d9f0f
rpcclient: use streaming for connect health
2021-03-12 11:35:42 -05:00
3d85c29445
Update content hash due to new field
2021-03-11 19:59:19 -07:00
d27208ce7c
Fixup more tests
2021-03-11 16:26:55 -07:00
be78d3c39a
Fixup protobufs and tests
2021-03-11 14:58:59 -07:00
237b41ac8f
Merge pull request #9672 from hashicorp/ca-force-skip-xc
...
connect/ca: Allow ForceWithoutCrossSigning for all providers
2021-03-11 11:49:15 -08:00
3de0b32cc5
Update service manager to store centrally configured upstreams
2021-03-11 11:37:21 -07:00
7a3625f58b
Add TransparentProxy opt to proxy definition
2021-03-11 11:37:21 -07:00
ac70808792
Restore old escape hatch alias
2021-03-11 11:36:35 -07:00
c30157d2f2
Turn Limits and PassiveHealthChecks into pointers
2021-03-11 11:04:40 -07:00
5dbeeee673
auto-updated agent/uiserver/bindata_assetfs.go from commit fa6687b7f
2021-03-11 09:34:21 +00:00
b98abb6f09
Update server-side config resolution and client-side merging
2021-03-10 21:05:11 -07:00
68148a1dae
finish moving UpstreamConfig and related fields to structs pkg
2021-03-10 21:04:13 -07:00
4877183bc6
Merge pull request #9797 from hashicorp/dnephin/state-index-node-id
...
state: convert nodes.ID to the new pattern of functional indexers
2021-03-10 17:34:23 -05:00
51ad94360b
state: move ConfigEntryKindName
...
Previously this type was defined in structs, but unlike the other types in structs this type
is not used by RPC requests. By moving it to state we can better indicate that this is not
an API type, but part of the state implementation.
2021-03-10 12:27:22 -05:00
5c5ba9564d
Merge pull request #9796 from hashicorp/dnephin/state-cleanup-catalog-index-oss
...
state: remove duplicate tableCheck indexes
2021-03-10 12:20:09 -05:00
97d4e872d3
Merge pull request #9851 from panascais-forks/fix-wan-ipv6-key
...
Fix advertise_addr_wan_ipv6 configuration key
2021-03-10 11:56:07 -05:00
94820e67a8
structs: remove EnterpriseMeta.GetNamespace
...
I added this recently without realizing that the method already existed and was named
NamespaceOrEmpty. Replace all calls to GetNamespace with NamespaceOrEmpty or NamespaceOrDefault
as appropriate.
2021-03-09 15:17:26 -05:00
848314782e
Merge pull request #9671 from hashicorp/streaming/terminating-gateway-events
...
state: Add terminating gateway events for streaming
2021-03-09 14:20:21 -05:00
97bc073bd9
state: adjust compare for catalog events
...
Document that this comparison should roughly match MatchesKey
Only sort by overrideKey or service name, but not both
Add namespace to the sort.
The client side also builds a map of these based on the namespace/node/service key, so the only order
that really matters is the ordering of register/dereigster events.
2021-03-09 14:00:36 -05:00
0d3bb68255
state: handle terminating gateway events properly in snapshot
...
Refactored out a function that can be used for both the snapshot and stream of events to translate
an event into an appropriate connect event.
Previously terminating gateway events would have used the wrong key in the snapshot, which would have
caused them to be filtered out later on.
Also removed an unused function, and some commented out code.
2021-03-09 14:00:35 -05:00
de3fba8ef3
Add remaining terminating gateway tests for namespaces
...
Co-Authored-By: Daniel Nephin <dnephin@hashicorp.com>
2021-03-09 14:00:35 -05:00
38aeb88908
Start to setup enterprise tests for terminating gateway streaming events.
...
Co-Authored-By: Kyle Havlovitz <kylehav@gmail.com>
2021-03-09 14:00:35 -05:00
d0b37f18f0
state: Add support for override of namespace
...
in MatchesKey
also tests for MatchesKey
Co-Authored-By: Kyle Havlovitz <kylehav@gmail.com>
2021-03-09 14:00:35 -05:00
ba59727337
state: update calls to ensureConfigEntryTxn
...
The EnterpriseMeta paramter was removed after this code was written, but before it merged.
Also the table name constant has changed.
2021-03-09 14:00:35 -05:00
730cc575e6
state: add 2 more test cases for terminate gateway streaming events
...
Co-Authored-By: Kyle Havlovitz <kylehav@gmail.com>
2021-03-09 14:00:34 -05:00
eadc8546a9
Added 6 new test cases for terminating gateway events
...
Co-Authored-By: Daniel Nephin <dnephin@hashicorp.com>
2021-03-09 14:00:34 -05:00
15b0d5f62b
state: Add two more tests for connect events with terminating gateways
...
And expand one test case to cover more.
Co-Authored-By: Kyle Havlovitz <kylehav@gmail.com>
2021-03-09 14:00:34 -05:00
abab373b89
state: Include the override key in the sorting of events
...
Co-Authored-By: Kyle Havlovitz <kylehav@gmail.com>
2021-03-09 14:00:34 -05:00
f31582624d
state: Add terminating gateway events on updating a config entry
...
Co-Authored-By: Daniel Nephin <dnephin@hashicorp.com>
2021-03-09 14:00:34 -05:00
f42a2ca8a3
state: add first terminating catalog catalog event
...
Health of a terminating gateway instance changes
- Generate an event for creating/destroying this instance of the terminating gateway,
duplicate it for each affected service
Co-Authored-By: Kyle Havlovitz <kylehav@gmail.com>
2021-03-09 14:00:33 -05:00
Paul Banks