Commit graph

3877 commits

Author SHA1 Message Date
Daniel Nephin 00ed2b243f
Merge pull request #10771 from hashicorp/dnephin/emit-telemetry-metrics-immediately
telemetry: improve cert expiry metrics
2021-11-01 18:31:03 -04:00
freddygv ecccf22fd7 Exclude default partition from GatewayKey string
This will behave the way we handle SNI and SPIFFE IDs, where the default
partition is excluded.

Excluding the default ensures that don't attempt to compare default.dc2
to dc2 in OSS.
2021-11-01 14:45:52 -06:00
freddygv d944e6ae3a Update GatewayKeys deduplication
Federation states data is only keyed on datacenter, so it cannot be
directly compared against keys for gateway groups.
2021-11-01 13:58:53 -06:00
freddygv ce43e8cf99 Store GatewayKey in proxycfg snapshot for re-use 2021-11-01 13:58:53 -06:00
freddygv 51c888a41a Update locality check in xds 2021-11-01 13:58:53 -06:00
freddygv 6657c88296 Update locality check in proxycfg 2021-11-01 13:58:53 -06:00
Daniel Nephin c706bf135c
Merge pull request #11340 from hashicorp/dnephin/ca-manager-provider
ca: split the Provider interface into Primary/Secondary
2021-11-01 14:11:15 -04:00
Daniel Nephin eaaceedf31
Merge pull request #11338 from hashicorp/dnephin/ca-manager-isolate-secondary
ca: clearly identify methods that are primary-only or secondary-only
2021-11-01 14:10:31 -04:00
Daniel Upton a620b6be2e
Support Check-And-Set deletion of config entries (#11419)
Implements #11372
2021-11-01 16:42:01 +00:00
Dhia Ayachi 4d763ef9e6
regenerate expired certs (#11462)
* regenerate expired certs

* add documentation to generate tests certificates
2021-11-01 11:40:16 -04:00
Jared Kirschner 6dfcbeceec
Merge pull request #11348 from kbabuadze/fix-answers-alt-domain
Fix answers for alt domain
2021-10-29 17:09:20 -04:00
R.B. Boyer d40d098321
agent: for various /v1/agent endpoints parse the partition parameter on the request (#11444)
Also update the corresponding CLI commands to send the parameter
appropriately.

NOTE: Behavioral changes are not happening in this PR.
2021-10-28 16:44:38 -05:00
R.B. Boyer 017e9d5ae4
agent: add a clone function for duplicating the serf lan configuration (#11443) 2021-10-28 16:11:26 -05:00
Daniel Nephin a8d6392ab5 Add tests for cert expiry metrics 2021-10-28 14:38:57 -04:00
Daniel Nephin 503dee2d80
Merge pull request #10671 from hashicorp/dnephin/fix-subscribe-test-flake
subscribe: improve TestSubscribeBackend_IntegrationWithServer_DeliversAllMessages
2021-10-28 12:57:09 -04:00
Evan Culver b3c92f22b1
connect: Remove support for Envoy 1.16 (#11354) 2021-10-27 18:51:35 -07:00
Evan Culver 98acbfa79c
connect: Add support for Envoy 1.20 (#11277) 2021-10-27 18:38:10 -07:00
freddygv 3dd21023bc Ensure partition-exports kind gets marshalled
The api module has decoding functions that rely on 'kind' being present
of payloads. This is so that we can decode into the appropriate api type
for the config entry.

This commit ensures that a static kind is marshalled in responses from
Consul's api endpoints so that the api module can decode them.
2021-10-27 15:01:26 -06:00
Daniel Nephin 0a19d7fd76 agent: move agent tls metric monitor to a more appropriate place
And add a test for it
2021-10-27 16:26:09 -04:00
Daniel Nephin 1b2144c982 telemetry: set cert expiry metrics to NaN on start
So that followers do not report 0, which would make alerting difficult.
2021-10-27 15:19:25 -04:00
Daniel Nephin a7fcf14c5c telemetry: fix cert expiry metrics by removing labels
These labels should be set by whatever process scrapes Consul (for
prometheus), or by the agent that receives them (for datadog/statsd).

We need to remove them here because the labels are part of the "metric
key", so we'd have to pre-declare the metrics with the labels. We could
do that, but that is extra work for labels that should be added from
elsewhere.

Also renames the closure to be more descriptive.
2021-10-27 15:19:25 -04:00
Daniel Nephin 4300daa2e6 telemetry: only emit leader cert expiry metrics on the servers 2021-10-27 15:19:25 -04:00
Daniel Nephin 9de725c17d telemetry: prevent stale values from cert monitors
Prometheus scrapes metrics from each process, so when leadership transfers to a different node
the previous leader would still be reporting the old cached value.

By setting NaN, I believe we should zero-out the value, so that prometheus should only consider the
value from the new leader.
2021-10-27 15:19:25 -04:00
Daniel Nephin 616cc9b6f8 telemetry: improve cert expiry metrics
Emit the metric immediately so that after restarting an agent, the new expiry time will be
emitted. This is particularly important when this metric is being monitored, because we want
the alert to resovle itself immediately.

Also fixed a bug that was exposed in one of these metrics. The CARoot can be nil, so we have
to handle that case.
2021-10-27 15:19:25 -04:00
Daniel Nephin 24951f0c7e subscribe: attempt to fix a flaky test
TestSubscribeBackend_IntegrationWithServer_DeliversAllMessages has been
flaking a few times. This commit cleans up the test a bit, and improves
the failure output.

I don't believe this actually fixes the flake, but I'm not able to
reproduce it reliably.

The failure appears to be that the event with Port=0 is being sent in
both the snapshot and as the first event after the EndOfSnapshot event.

Hopefully the improved logging will show us if these are really
duplicate events, or actually different events with different indexes.
2021-10-27 15:09:09 -04:00
Freddy ae76144f55
Merge pull request #11435 from hashicorp/ent-authorizer-refactor
[OSS] Export ACLs refactor
2021-10-27 13:04:40 -06:00
Freddy 520bda999b
Merge pull request #11432 from hashicorp/ap/exports-mgw
[OSS] Update mesh gateways to handle partitions
2021-10-27 12:54:53 -06:00
freddygv 592965d61e Rework acl exports interface 2021-10-27 12:50:39 -06:00
Freddy 9bbeea0432
Merge pull request #11433 from hashicorp/exported-service-acls
[OSS] acl: Expand ServiceRead and NodeRead to account for partition exports
2021-10-27 12:48:08 -06:00
freddygv 05f91bd2b8 Update comments 2021-10-27 12:36:44 -06:00
Freddy d8ae915160
Merge pull request #11431 from hashicorp/ap/exports-proxycfg
[OSS] Update partitioned mesh gw handling for connect proxies
2021-10-27 11:27:43 -06:00
Freddy 8e23a6a0cc
Merge pull request #11416 from hashicorp/ap/exports-update
Rename service-exports to partition-exports
2021-10-27 11:27:31 -06:00
freddygv 40271beb38 Fixup partitions assertion 2021-10-27 11:15:25 -06:00
freddygv 67412ac5e7 Fixup imports 2021-10-27 11:15:25 -06:00
freddygv 4de3537391 Split up locality check from hostname check 2021-10-27 11:15:25 -06:00
freddygv 9769b31641 Move the exportingpartitions constant to enterprise 2021-10-27 11:15:25 -06:00
freddygv 0391a65772 Replace default partition check 2021-10-27 11:15:25 -06:00
freddygv ee45ac9dc5 PR comments 2021-10-27 11:15:25 -06:00
freddygv f99946553a Leave todo about default name 2021-10-27 11:15:25 -06:00
freddygv 9d375ad6d2 Add oss impl of registerEntCache 2021-10-27 11:15:25 -06:00
freddygv 183849416b Register the ExportingPartitions cache type 2021-10-27 11:15:25 -06:00
freddygv 8b5a9369eb Account for partitions in xds gen for mesh gw
This commit avoids skipping gateways in remote partitions of the local
DC when generating listeners/clusters/endpoints.
2021-10-27 11:15:25 -06:00
freddygv d1d513b1b3 Account for partition in SNI for gateways 2021-10-27 11:15:25 -06:00
freddygv 4f0432be5e Update xds pkg to account for GatewayKey 2021-10-27 09:03:56 -06:00
freddygv f3f15640a9 Update mesh gateway proxy watches for partitions
This commit updates mesh gateway watches for cross-partitions
communication.

* Mesh gateways are keyed by partition and datacenter.

* Mesh gateways will now watch gateways in partitions that export
services to their partition.

* Mesh gateways in non-default partitions will not have cross-datacenter
watches. They are not involved in traditional WAN federation.
2021-10-27 09:03:56 -06:00
freddygv af662c8c1c Avoid mixing named and unnamed params 2021-10-26 23:42:25 -06:00
freddygv 1de62bb0a2 Avoid passing nil config pointer 2021-10-26 23:42:25 -06:00
freddygv 4a2e40aa3c Avoid panic on nil partitionAuthorizer config
partitionAuthorizer.config can be nil if it wasn't provided on calls to
newPartitionAuthorizer outside of the ACLResolver. This usage happens
often in tests.

This commit: adds a nil check when the config is going to be used,
updates non-test usage of NewPolicyAuthorizerWithDefaults to pass a
non-nil config, and dettaches setEnterpriseConf from the ACLResolver.
2021-10-26 23:42:25 -06:00
freddygv 015d85cd74 Update NodeRead for partition-exports
When issuing cross-partition service discovery requests, ACL filtering
often checks for NodeRead privileges. This is because the common return
type is a CheckServiceNode, which contains node data.
2021-10-26 23:42:11 -06:00
Kyle Havlovitz afb0976eac acl: pass PartitionInfo through ent ACLConfig 2021-10-26 23:41:52 -06:00
Kyle Havlovitz 56d1858c4a acl: Expand ServiceRead logic to look at service-exports for cross-partition 2021-10-26 23:41:32 -06:00
freddygv 4737ad118d Swap in structs.EqualPartitions for cmp 2021-10-26 23:36:01 -06:00
freddygv 1bade08f91 Replace Split with SplitN 2021-10-26 23:36:01 -06:00
freddygv 3966677aaf Finish removing useInDatacenter 2021-10-26 23:36:01 -06:00
freddygv 69476221c1 Update XDS for sidecars dialing through gateways 2021-10-26 23:35:48 -06:00
freddygv ea311d2e47 Configure sidecars to watch gateways in partitions
Previously the datacenter of the gateway was the key identifier, now it
is the datacenter and partition.

When dialing services in other partitions or datacenters we now watch
the appropriate partition.
2021-10-26 23:35:37 -06:00
freddygv feaebde1f1 Remove useInDatacenter from disco chain requests
useInDatacenter was used to determine whether the mesh gateway mode of
the upstream should be returned in the discovery chain target. This
commit makes it so that the mesh gateway mode is returned every time,
and it is up to the caller to decide whether mesh gateways should be
watched or used.
2021-10-26 23:35:21 -06:00
R.B. Boyer e27e58c6cc
agent: refactor the agent delegate interface to be partition friendly (#11429) 2021-10-26 15:08:55 -05:00
Chris S. Kim 27f8a85664
agent: Ensure partition is considered in agent endpoints (#11427) 2021-10-26 15:20:57 -04:00
Konstantine 2f9ee8e558 remove spaces 2021-10-26 12:38:13 -04:00
Konstantine be14f6da90 fix altDomain responses for services where address is IP, added tests 2021-10-26 12:38:13 -04:00
Konstantine eec9d66e22 fix encodeIPAsFqdn to return alt-domain when requested, added test case 2021-10-26 12:38:12 -04:00
Konstantine 9d6797a463 fixed altDomain response for NS type queries, and added test 2021-10-26 12:38:12 -04:00
Konstantine 0735e12412 edited TestDNS_AltDomains_Service to test responses for altDomains, and added TXT additional section check 2021-10-26 12:38:12 -04:00
Konstantine 8972e093d9 fixed alt-domain answer for SRV records, and TXT records in additional section 2021-10-26 12:38:12 -04:00
Chris S. Kim 3f736467e6
ui: Pass primary dc through to uiserver (#11317)
Co-authored-by: John Cowen <johncowen@users.noreply.github.com>
2021-10-26 10:30:17 -04:00
freddygv 83d4d0e108 Remove outdated partition label from test 2021-10-25 18:47:02 -06:00
freddygv c3e381b4c1 Rename service-exports to partition-exports
Existing config entries prefixed by service- are specific to individual
services. Since this config entry applies to partitions it is being
renamed.

Additionally, the Partition label was changed to Name because using
Partition at the top-level and in the enterprise meta was leading to the
enterprise meta partition being dropped by msgpack.
2021-10-25 17:58:48 -06:00
Daniel Nephin f24bad2a52
Merge pull request #11232 from hashicorp/dnephin/acl-legacy-remove-docs
acl: add docs and changelog for the removal of the legacy ACL system
2021-10-25 18:38:00 -04:00
Daniel Nephin f7cdd210fe Update agent/consul/acl_client.go
Co-authored-by: Freddy <freddygv@users.noreply.github.com>
2021-10-25 17:25:14 -04:00
Daniel Nephin 732b841dd7 state: remove support for updating legacy ACL tokens 2021-10-25 17:25:14 -04:00
Daniel Nephin 76b007dacd acl: remove init check for legacy anon token
This token should always already be migrated from a previous version.
2021-10-25 17:25:14 -04:00
Daniel Nephin 8ae6ee4e36 acl: remove legacy parameter to ACLDatacenter
It is no longer used now that legacy ACLs have been removed.
2021-10-25 17:25:14 -04:00
Daniel Nephin d778113773 acl: remove ACLTokenTypeManagement 2021-10-25 17:25:14 -04:00
Daniel Nephin 2f0eba1980 acl: remove ACLTokenTypeClient,
along with the last test referencing it.
2021-10-25 17:25:14 -04:00
Daniel Nephin 88c6aeea34 acl: remove legacy arg to store.ACLTokenSet
And remove the tests for legacy=true
2021-10-25 17:25:14 -04:00
Daniel Nephin b31a7fc498 acl: remove EmbeddedPolicy
This method is no longer. It only existed for legacy tokens, which are no longer supported.
2021-10-25 17:25:14 -04:00
Daniel Nephin ceaa36f983 acl: remove tests for resolving legacy tokens
The code for this was already removed, which suggests this is not actually testing what it claims.

I'm guessing these are still resolving because the tokens are converted to non-legacy tokens?
2021-10-25 17:25:14 -04:00
Daniel Nephin a46e3bd2fc acl: stop replication on leadership lost
It seems like this was missing. Previously this was only called by init of ACLs during an upgrade.
Now that legacy ACLs are  removed, nothing was calling stop.

Also remove an unused method from client.
2021-10-25 17:24:12 -04:00
Daniel Nephin 15cd8c7ab8 Remove incorrect TODO 2021-10-25 17:20:06 -04:00
Daniel Nephin 589b238374 acl: move the legacy ACL struct to the one package where it is used
It is now only used for restoring snapshots. We can remove it in phase 2.
2021-10-25 17:20:06 -04:00
Daniel Nephin 0ba5d0afcd acl: remove most of the rest of structs/acl_legacy.go 2021-10-25 17:20:06 -04:00
Paul Banks ab5cdce760
Merge pull request #11163 from hashicorp/feature/ingress-tls-mixed
Add support for enabling connect-based ingress TLS per listener.
2021-10-25 21:36:01 +01:00
FFMMM 6433a57d3c
fix autopilot_failure_tolerance, add autopilot metrics test case (#11399)
Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>
2021-10-25 10:55:59 -07:00
FFMMM 67a624a49f
use *telemetry.MetricsPrefix as prometheus.PrometheusOpts.Name (#11290)
Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>
2021-10-21 13:33:01 -07:00
Dhia Ayachi 75f69a98a2
fix leadership transfer on leave suggestions (#11387)
* add suggestions

* set isLeader to false when leadership transfer succeed
2021-10-21 14:02:26 -04:00
Dhia Ayachi 2d1ac1f7d0
try to perform a leadership transfer when leaving (#11376)
* try to perform a leadership transfer when leaving

* add a changelog
2021-10-21 12:44:31 -04:00
Kyle Havlovitz 752a285552 Add new service-exports config entry 2021-10-20 12:24:18 -07:00
Jared Kirschner 716b05f934
Merge pull request #11293 from bisakhmondal/service_filter
expression validation of service-resolver subset filter
2021-10-20 08:57:37 -04:00
Paul Banks 4808b97d9c Rebase and rebuild golden files for Envoy version bump 2021-10-19 21:37:58 +01:00
Paul Banks ff405d35c7 Refactor resolveListenerSDSConfig to pass in whole config 2021-10-19 20:58:29 +01:00
Paul Banks 5c8702b182 Add support for enabling connect-based ingress TLS per listener. 2021-10-19 20:58:28 +01:00
R.B. Boyer 55dd52cb17
acl: small OSS refactors to help ensure that auth methods with namespace rules work with partitions (#11323) 2021-10-14 15:38:05 -05:00
freddygv f76fddb28e Use stored entmeta to fill authzContext 2021-10-14 08:57:40 -06:00
freddygv bdf3e951f8 Ensure partition is handled by auto-encrypt 2021-10-14 08:32:45 -06:00
FFMMM bb228ab165
fix: only add prom autopilot gauges to servers (#11241)
Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>
2021-10-13 09:25:30 -07:00
Chris S. Kim 0a6d683c84
Update Intentions.List with partitions (#11299) 2021-10-13 10:47:12 -04:00
R.B. Boyer 3e8ece97a8
acl: fix bug in 'consul members' filtering with partitions (#11263) 2021-10-13 09:18:16 -05:00
Bisakh Mondal 929ad1e80f
add service resolver subset filter validation 2021-10-13 02:56:04 +05:30
Connor 2cd80e5f66
Merge pull request #11222 from hashicorp/clly/service-mesh-metrics
Start tracking connect service mesh usage metrics
2021-10-11 14:35:03 -05:00