Commit graph

3599 commits

Author SHA1 Message Date
Chris S. Kim 3fb797382b
Sync enterprise changes to oss (#10994)
This commit updates OSS with files for enterprise-specific admin partitions feature work
2021-09-08 11:59:30 -04:00
Kyle Havlovitz a7b5a5d1b4
Merge pull request #10984 from hashicorp/mesh-resource
acl: adding a new mesh resource
2021-09-07 15:06:20 -07:00
Dhia Ayachi 96d7842118
partition dicovery chains (#10983)
* partition dicovery chains

* fix default partition for OSS
2021-09-07 16:29:32 -04:00
R.B. Boyer 4206f585f0 acl: adding a new mesh resource 2021-09-03 09:12:03 -04:00
Dhia Ayachi 72391dc99c
try to infer command partition from node partition (#10981) 2021-09-03 08:37:23 -04:00
Dhia Ayachi eb19271fd7
add partition to SNI when partition is non default (#10917) 2021-09-01 10:35:39 -04:00
Freddy 11672defaf
connect: update envoy supported versions to latest patch release
(#10961)

Relevant advisory: 
https://github.com/envoyproxy/envoy/security/advisories/GHSA-6g4j-5vrw-2m8h
2021-08-31 10:39:18 -06:00
Evan Culver 93f94ac24f
rpc: authorize raft requests (#10925) 2021-08-26 15:04:32 -07:00
hc-github-team-consul-core a758581ab6 auto-updated agent/uiserver/bindata_assetfs.go from commit eeeb91bea 2021-08-26 18:13:08 +00:00
Chris S. Kim 86de20c975
ent->oss test fix (#10926) 2021-08-26 14:06:49 -04:00
hc-github-team-consul-core 5c67517647 auto-updated agent/uiserver/bindata_assetfs.go from commit a907e1d87 2021-08-26 18:02:18 +00:00
hc-github-team-consul-core d9022ce788 auto-updated agent/uiserver/bindata_assetfs.go from commit a0b0ed2bc 2021-08-26 16:06:09 +00:00
Chris S. Kim efbdf7e117
api: expose upstream routing configurations in topology view (#10811)
Some users are defining routing configurations that do not have associated services. This commit surfaces these configs in the topology visualization. Also fixes a minor internal bug with non-transparent proxy upstream/downstream references.
2021-08-25 15:20:32 -04:00
R.B. Boyer 6b5a58de50
acl: some acl authz refactors for nodes (#10909) 2021-08-25 13:43:11 -05:00
hc-github-team-consul-core c95ec5007d auto-updated agent/uiserver/bindata_assetfs.go from commit a777b0a9b 2021-08-25 13:46:51 +00:00
hc-github-team-consul-core 9b2dd8b155 auto-updated agent/uiserver/bindata_assetfs.go from commit 8192dde48 2021-08-25 11:39:14 +00:00
R.B. Boyer a84f5fa25d
grpc: ensure that streaming gRPC requests work over mesh gateway based wan federation (#10838)
Fixes #10796
2021-08-24 16:28:44 -05:00
hc-github-team-consul-core 6b574abc89 auto-updated agent/uiserver/bindata_assetfs.go from commit 05a28c311 2021-08-24 16:04:24 +00:00
freddygv 79e181be73 Avoid passing zero value into variadic 2021-08-20 17:40:33 -06:00
freddygv ed79e38a36 Update comment for test function 2021-08-20 17:40:33 -06:00
freddygv b1050e4229 Update prepared query cluster SAN validation
Previously SAN validation for prepared queries was broken because we
validated against the name, namespace, and datacenter for prepared
queries.

However, prepared queries can target:

- Services with a name that isn't their own
- Services in multiple datacenters

This means that the SpiffeID to validate needs to be based on the
prepared query endpoints, and not the prepared query's upstream
definition.

This commit updates prepared query clusters to account for that.
2021-08-20 17:40:33 -06:00
freddygv 1f192eb7d9 Fixup proxy config test fixtures
- The TestNodeService helper created services with the fixed name "web",
and now that name is overridable.

- The discovery chain snapshot didn't have prepared query endpoints so
the endpoints tests were missing data for prepared queries
2021-08-20 17:38:57 -06:00
R.B. Boyer 60591d55f7
agent: add partition labels to catalog API metrics where appropriate (#10890) 2021-08-20 15:09:39 -05:00
R.B. Boyer b6be94e7fa
fixing various bits of enterprise meta plumbing to be more correct (#10889) 2021-08-20 14:34:23 -05:00
Dhia Ayachi f766b6dff7
oss portion of ent #1069 (#10883) 2021-08-20 12:57:45 -04:00
R.B. Boyer d730298f59
state: partition the nodes.uuid and nodes.meta indexes as well (#10882) 2021-08-19 16:17:59 -05:00
R.B. Boyer 61f1c01b83
agent: ensure that most agent behavior correctly respects partition configuration (#10880) 2021-08-19 15:09:42 -05:00
Daniel Nephin 4a0ae4048d
Merge pull request #10849 from hashicorp/dnephin/contrib-doc-xds-auth
xds: document how authorization works
2021-08-18 13:25:16 -04:00
R.B. Boyer e565409c6a
state: partition the usage metrics subsystem (#10867) 2021-08-18 09:27:15 -05:00
Daniel Nephin 9df2464c7c xds: document how authorization works 2021-08-17 19:26:34 -04:00
R.B. Boyer 1cef3c99c2
state: adjust streaming event generation to account for partitioned nodes (#10860)
Also re-enabled some tests that had to be disabled in the prior PR.
2021-08-17 16:49:26 -05:00
R.B. Boyer e50e13d2ab
state: partition nodes and coordinates in the state store (#10859)
Additionally:

- partitioned the catalog indexes appropriately for partitioning
- removed a stray reference to a non-existent index named "node.checks"
2021-08-17 13:29:39 -05:00
Daniel Nephin 5a82859ee1 acl: small improvements to ACLResolver disable due to RPC error
Remove the error return, so that not handling is not reported as an
error by errcheck. It was returning the error passed as an arg
unmodified so there is no reason to return the same value that was
passed in.

Remove the term upstreams to remove any confusion with the term used in
service mesh.

Remove the AutoDisable field, and replace it with the TTL value, using 0
to indicate the setting is turned off.

Replace "not Before" with "After".

Add some test coverage to show the behaviour is still correct.
2021-08-17 13:34:18 -04:00
Daniel Nephin 09ae0ab94a acl: make ACLDisabledTTL a constant
This field was never user-configurable. We always overwrote the value with 120s from
NonUserSource. However, we also never copied the value from RuntimeConfig to consul.Config,
So the value in NonUserSource was always ignored, and we used the default value of 30s
set by consul.DefaultConfig.

All of this code is an unnecessary distraction because a user can not actually configure
this value.

This commit removes the fields and uses a constant value instad. Someone attempting to set
acl.disabled_ttl in their config will now get an error about an unknown field, but previously
the value was completely ignored, so the new behaviour seems more correct.

We have to keep this field in the AutoConfig response for backwards compatibility, but the value
will be ignored by the client, so it doesn't really matter what value we set.
2021-08-17 13:34:18 -04:00
Daniel Nephin a8bc964241 Fix test failures
Tests only specified one of the fields, but in production we copy the
value from a single place, so we can do the same in tests.

The AutoConfig test broke because of the problem noticed in a previous
commit. The DisabledTTL is not wired up properly so it reports 0s here.
Changed the test to use an explicit value.
2021-08-17 13:32:52 -04:00
Daniel Nephin 0d69b49f41 config: remove ACLResolver settings from RuntimeConfig 2021-08-17 13:32:52 -04:00
Daniel Nephin 75baa22e64 acl: remove ACLResolver config fields from consul.Config 2021-08-17 13:32:52 -04:00
Daniel Nephin 454f62eacc acl: replace ACLResolver.Config with its own struct
This is step toward decoupling ACLResolver from the agent/consul
package.
2021-08-17 13:32:52 -04:00
Daniel Nephin 5e5ad62679 acl: remove ACLRulesTranslateLegacyToken API endpoint 2021-08-17 13:10:02 -04:00
Daniel Nephin be0358df02 acl: remove legacy bootstrap
Return an explicit error from the RPC, and remove the flag from the HTTP API.
2021-08-17 13:10:00 -04:00
Daniel Nephin d877673268 agent: update some tests that were using legacy ACL endpoints
The tests were updated to use the new ACL endpoints now that the legacy ones have been removed.
2021-08-17 13:09:30 -04:00
Daniel Nephin 10791b007d http: update legacy ACL endpoints to return an error
Also move a test for the ACLReplicationStatus endpoint into the correct file.
2021-08-17 13:09:29 -04:00
Daniel Nephin 4f54d9708c acl: add some notes about removing legacy ACL system 2021-08-17 13:08:29 -04:00
Daniel Nephin e4c6bee7e6
Merge pull request #10792 from hashicorp/dnephin/rename-authz-vars
acl: use authz consistently as the variable name for an acl.Authorizer
2021-08-17 13:07:17 -04:00
Daniel Nephin 7f71a672f3
Merge pull request #10807 from hashicorp/dnephin/remove-acl-datacenter
config: remove ACLDatacenter
2021-08-17 13:07:09 -04:00
Daniel Nephin 608b291565 acl: use authz consistently as the variable name for an acl.Authorizer
Follow up to https://github.com/hashicorp/consul/pull/10737#discussion_r682147950

Renames all variables for acl.Authorizer to use `authz`. Previously some
places used `rule` which I believe was an old name carried over from the
legacy ACL system.

A couple places also used authorizer.

This commit also removes another couple of authorizer nil checks that
are no longer necessary.
2021-08-17 12:14:10 -04:00
hc-github-team-consul-core e1da3da0e2 auto-updated agent/uiserver/bindata_assetfs.go from commit ae9c31338 2021-08-16 16:10:17 +00:00
Kyle Havlovitz 470558708e
Merge pull request #10843 from hashicorp/partitions/rename-default
oss: Rename default partition
2021-08-12 14:45:53 -07:00
Kyle Havlovitz 98969c018a oss: Rename default partition 2021-08-12 14:31:37 -07:00
Daniel Nephin 7c865d03ac proxycfg: Lookup the agent token as a default
When no ACL token is provided with the service registration.
2021-08-12 15:51:34 -04:00
Daniel Nephin d189524e71 proxycfg: Add a test to show the bug
When a token is not provided at registration, the agent token is not being used.
2021-08-12 15:47:59 -04:00
Mike Morris 86d76cb099
deps: upgrade gogo-protobuf to v1.3.2 (#10813)
* deps: upgrade gogo-protobuf to v1.3.2

* go mod tidy using go 1.16

* proto: regen protobufs after upgrading gogo/protobuf

Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>
2021-08-12 14:05:46 -04:00
Mark Anderson 03a3ec2b55
Fixup to support unix domain socket via command line (#10758)
Missed the need to add support for unix domain socket config via
api/command line. This is a variant of the problems described in
it is easy to drop one.

Signed-off-by: Mark Anderson <manderson@hashicorp.com>
2021-08-12 10:05:22 -07:00
hc-github-team-consul-core f02ea91a8b auto-updated agent/uiserver/bindata_assetfs.go from commit ab6a67520 2021-08-11 17:05:51 +00:00
Daniel Nephin 364ef3d052 server: remove defaulting of PrimaryDatacenter
The constructor for Server is not at all the appropriate place to be setting default
values for a config struct that was passed in.

In production this value is always set from agent/config. In tests we should set the
default in a test helper.
2021-08-06 18:45:24 -04:00
Daniel Nephin 87fb26fd65
Merge pull request #10612 from bigmikes/acl-replication-fix
acl: acl replication routine to report the last error message
2021-08-06 18:29:51 -04:00
Daniel Nephin 047abdd73c acl: remove ACLDatacenter
This field has been unnecessary for a while now. It was always set to the same value
as PrimaryDatacenter. So we can remove the duplicate field and use PrimaryDatacenter
directly.

This change was made by GoLand refactor, which did most of the work for me.
2021-08-06 18:27:00 -04:00
Giulio Micheloni 5c34a48d45 String type instead of error type and changelog. 2021-08-06 22:35:27 +01:00
Daniel Nephin 9435118179 acl: remove Server.ResolveTokenIdentityAndDefaultMeta
This method suffered from similar naming to a couple other methods on Server, and had not great
re-use (2 callers). By copying a few of the lines into one of the callers we can move the
implementation into the second caller.

Once moved, we can see that ResolveTokenAndDefaultMeta is identical in both Client and Server, and
likely should be further refactored, possibly into ACLResolver.

This change is being made to make ACL resolution easier to trace.
2021-08-05 15:20:13 -04:00
Daniel Nephin 25f40de163 acl: remove Server.ResolveTokenToIdentityAndAuthorizer
This method was an alias for ACLResolver.ResolveTokenToIdentityAndAuthorizer. By removing the
method that does nothing the code becomes easier to trace.
2021-08-05 15:20:13 -04:00
Daniel Nephin 695963acb7 acl: recouple acl filtering from ACLResolver
ACL filtering only needs an authorizer and a logger. We can decouple filtering from
the ACLResolver by passing in the necessary logger.

This change is being made in preparation for moving the ACLResolver into an acl package
2021-08-05 15:20:13 -04:00
Daniel Nephin ba2f9a65d1 acl: remove unused error return
filterACLWithAuthorizer could never return an error. This change moves us a little bit
closer to being able to enable errcheck and catch problems caused by unhandled error
return values.
2021-08-05 15:20:13 -04:00
Daniel Nephin c80b9565e2 acl: rename acl.Authorizer vars to authz
For consistency
2021-08-05 15:19:47 -04:00
Daniel Nephin 37c67cb280 acl: move vet functions
These functions are moved to the one place they are called to improve code locality.

They are being moved out of agent/consul/acl.go in preparation for moving
ACLResolver to an acl package.
2021-08-05 15:19:24 -04:00
Daniel Nephin c8eedabc7c acl: move vetRegisterWithACL and vetDeregisterWithACL
These functions are used in only one place. Move the functions next to their one caller
to improve code locality.

This change is being made in preparation for moving the ACLResolver into an
acl package. The moved functions were previously in the same file as the ACLResolver.
By moving them out of that file we may be able to move the entire file
with fewer modifications.
2021-08-05 15:17:54 -04:00
Daniel Nephin b223c2bc25
Merge pull request #10770 from hashicorp/dnephin/log-cert-expiration
telemetry: add log message when certs are about to expire
2021-08-05 15:17:20 -04:00
Daniel Nephin c866f1041a
Merge pull request #10793 from hashicorp/dnephin/acl-intentions
acl: small cleanup of a couple Authorization flows
2021-08-05 15:16:49 -04:00
Dhia Ayachi 40baf98159
defer setting the state before returning to avoid stuck in INITIALIZING state (#10630)
* defer setting the state before returning to avoid being stuck in `INITIALIZING` state

* add changelog

* move comment with the right if statement

* ca: report state transition error from setSTate

* update comment to reflect state transition

Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>
2021-08-05 14:51:19 -04:00
Daniel Nephin 79ab48ef81
Merge pull request #10768 from hashicorp/dnephin/agent-tls-cert-expiration-metric
telemetry: add Agent TLS Certificate expiration metric
2021-08-04 18:42:02 -04:00
Daniel Nephin 0ca9e875e2 acl: remove special handling of services in txn_endpoint
Follow up to: https://github.com/hashicorp/consul/pull/10738#discussion_r680190210

Previously we were passing an Authorizer that would always allow the
operation, then later checking the authorization using vetServiceTxnOp.

On the surface this seemed strange, but I think it was actually masking
a bug as well. Over time `servicePreApply` was changed to add additional
authorization for `service.Proxy.DestinationServiceName`, but because
we were passing a nil Authorizer, that authorization was not handled on
the txn_endpoint.

`TxnServiceOp.FillAuthzContext` has some special handling in enterprise,
so we need to make sure to continue to use that from the Txn endpoint.

This commit removes the `vetServiceTxnOp` function, and passes in the
`FillAuthzContext` function so that `servicePreApply` can be used by
both the catalog and txn endpoints. This should be much less error prone
and prevent bugs like this in the future.
2021-08-04 18:32:20 -04:00
hc-github-team-consul-core ef162f8390 auto-updated agent/uiserver/bindata_assetfs.go from commit bcd53e73a 2021-08-04 22:27:44 +00:00
Daniel Nephin f6d5a85561 acl: move check for Intention.DestinationName into Authorizer
Follow up to https://github.com/hashicorp/consul/pull/10737#discussion_r680134445

Move the check for the Intention.DestinationName into the Authorizer to remove the
need to check what kind of Authorizer is being used.

It sounds like this check is only for legacy ACLs, so is probably just a safeguard
.
2021-08-04 18:06:44 -04:00
Daniel Nephin 3dc113ada6
Merge pull request #10738 from hashicorp/dnephin/remove-authorizer-nil-checks-2
acl: remove the last of the authz == nil checks
2021-08-04 17:41:40 -04:00
Daniel Nephin 2e9aa91256
Merge pull request #10737 from hashicorp/dnephin/remove-authorizer-nil-checks
acl: remove authz == nil checks
2021-08-04 17:39:34 -04:00
Daniel Nephin 210a850353 telemetry: add log message when certs are about to expire 2021-08-04 14:18:59 -04:00
Daniel Nephin 13aa7b70d5 telemetry: fix a couple bugs in cert expiry metrics
1. do not emit the metric if Query fails
2. properly check for PrimaryUsersIntermediate, the logic was inverted

Also improve the logging by including the metric name in the log message
2021-08-04 13:51:44 -04:00
Daniel Nephin 1673b3a68c telemetry: add a metric for agent TLS cert expiry 2021-08-04 13:51:44 -04:00
Dhia Ayachi 6ed6966a1f
fix state index for CAOpSetRootsAndConfig op (#10675)
* fix state index for `CAOpSetRootsAndConfig` op

* add changelog

* Update changelog

Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>

* remove the change log as it's not needed

Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>
2021-08-04 13:07:49 -04:00
hc-github-team-consul-core 4b2ada0dcc auto-updated agent/uiserver/bindata_assetfs.go from commit 8ad1ab9c0 2021-08-04 16:47:13 +00:00
Evan Culver 57aabe3455
checks: Add Interval and Timeout to API response (#10717) 2021-08-03 15:26:49 -07:00
Daniel Nephin 953c9bee4f acl: Remove the remaining authz == nil checks
These checks were a bit more involved. They were previously skipping some code paths
when the authorizer was nil. After looking through these it seems correct to remove the
authz == nil check, since it will never evaluate to true.
2021-07-30 14:55:35 -04:00
Daniel Nephin e4821a58ee acl: remove acl == nil checks 2021-07-30 14:28:19 -04:00
Daniel Nephin fbaeac9ecf acl: remove authz == nil checks
These case are already impossible conditions, because most of these functions already start
with a check for ACLs being disabled. So the code path being removed could never be reached.

The one other case (ConnectAuthorized) was already changed in a previous commit. This commit
removes an impossible branch because authz == nil can never be true.
2021-07-30 13:58:35 -04:00
Daniel Nephin b6d9d0d9f7 acl: remove many instances of authz == nil 2021-07-30 13:58:35 -04:00
Daniel Nephin bbc05ae869 agent: remove unused agent methods
These methods are no longer used. Remove the methods, and update the
tests to use actual method used by production code.

Also removes the 'authz == nil' check is no longer a possible code path
now that we are returning a non-nil acl.Authorizer when ACLs are disabled.
2021-07-30 13:58:35 -04:00
Daniel Nephin 2503f27a36 acl: remove rule == nil checks 2021-07-30 13:58:35 -04:00
hc-github-team-consul-core 701d4ffef0 auto-updated agent/uiserver/bindata_assetfs.go from commit 2ee501be8 2021-07-30 17:58:27 +00:00
Daniel Nephin 475fec5670
Merge pull request #10632 from hashicorp/pairing/acl-authorizer-when-acl-disabled
acls: Update ACL authorizer to return meaningful permission when ACLs are disabled
2021-07-30 13:22:55 -04:00
Evan Culver 241b6429c3 Fix intention endpoint test 2021-07-30 12:58:45 -04:00
Daniel Nephin 9b41e7287f acl: use acl.ManangeAll when ACLs are disabled
Instead of returning nil and checking for nilness

Removes a bunch of nil checks, and fixes one test failures.
2021-07-30 12:58:24 -04:00
Blake Covarrubias f97e843c61 Add OSS changes for specifying audit log permission mode 2021-07-30 09:58:11 -07:00
Daniel Nephin f2f5aba1bf
Merge pull request #10707 from hashicorp/dnephin/streaming-setup-default-timeout
streaming: set default query timeout
2021-07-28 18:29:28 -04:00
Daniel Nephin 057e8320f9 streaming: set a default timeout
The blocking query backend sets the default value on the server side.
The streaming backend does not using blocking queries, so we must set the timeout on
the client.
2021-07-28 17:50:00 -04:00
hc-github-team-consul-core f39d36d346 auto-updated agent/uiserver/bindata_assetfs.go from commit eb5512fb7 2021-07-27 21:39:22 +00:00
Chris S. Kim 33d7d48767
sync enterprise files with oss (#10705) 2021-07-27 17:09:59 -04:00
Daniel Nephin cfc829275c http: don't log an error if the request is cancelled
Now that we have at least one endpoint that uses context for cancellation we can
encounter this scenario where the returned error is a context.Cancelled or
context.DeadlineExceeded.

If the request.Context().Err() is not nil, then we know the request itself was cancelled, so
we can log a different message at Info level, instad of the error.
2021-07-27 17:06:59 -04:00
Daniel Nephin bad2c4ef67
Merge pull request #10399 from hashicorp/dnephin/debug-stream-metrics
debug: use the new metrics stream in debug command
2021-07-27 13:23:15 -04:00
Daniel Nephin 7d24564ff0 http: add tests for AgentMetricsStream 2021-07-26 17:53:33 -04:00
Daniel Nephin cf2e25c6bb http: emit indented JSON in the metrics stream endpoint
To remove the need to decode and re-encode in the CLI
2021-07-26 17:53:33 -04:00
Daniel Nephin d716f709fd debug: use the new metrics stream in debug command 2021-07-26 17:53:32 -04:00