Fix a bug that wrongly trims domains when there is an overlap with DC name (#17160)
* Fix a bug that wrongly trims domains when there is an overlap with DC name
Before this change, when DC name and domain/alt-domain overlap, the domain name incorrectly trimmed from the query.
Example:
Given: datacenter = dc-test, alt-domain = test.consul.
Querying for "test-node.node.dc-test.consul" will faile, because the
code was trimming "test.consul" instead of just ".consul"
This change, fixes the issue by adding dot (.) before trimming
* trimDomain: ensure domain trimmed without modyfing original domains
* update changelog
---------
Co-authored-by: Alex Simenduev <shamil.si@gmail.com>
* backport of commit 57bb6f3d729e4d76d1043efa2fa6a46137398d32
* backport of commit b2dad880653285a975795e89b0d77a6ea2fa60f1
* backport of commit 753d3c0d3f4797b6cf2d3490df996dffa8e885de
---------
Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
* no-op commit due to failed cherry-picking
* add a conditional around setting LANFilter.AllSegments to make sure it is valid (#18139)
This is to correct a code problem because this assumes all segments, but
when you get to Enterprise, you can be in partition that is not the
default partition, in which case specifying all segments does not
validate and fails. This is to correct the setting of this filter with
`AllSegments` to `true` to only occur when in the the `default`
partition.
<!--
* In the case of bugs, describe how to replicate
* If any manual tests were done, document the steps and the conditions
to replicate
* Call out any important/ relevant unit tests, e2e tests or integration
tests you have added or are adding
-->
<!--
Include any links here that might be helpful for people reviewing your
PR (Tickets, GH issues, API docs, external benchmarks, tools docs, etc).
If there are none, feel free to delete this section.
Please be mindful not to leak any customer or confidential information.
HashiCorp employees may want to use our internal URL shortener to
obfuscate links.
-->
* [ ] updated test coverage
* [ ] external facing docs updated
* [ ] appropriate backport labels added
* [ ] not a security concern
---------
Co-authored-by: temp <temp@hashicorp.com>
Co-authored-by: John Murret <john.murret@hashicorp.com>
## Backport
This PR is auto-generated from #17754 to be assessed for backporting due
to the inclusion of the label backport/1.16.
🚨
>**Warning** automatic cherry-pick of commits failed. If the first
commit failed,
you will see a blank no-op commit below. If at least one commit
succeeded, you
will see the cherry-picked commits up to, _not including_, the commit
where
the merge conflict occurred.
The person who merged in the original PR is:
@WenInCode
This person should manually cherry-pick the original PR into a new
backport PR,
and close this one when the manual backport PR is merged in.
> merge conflict error: unable to process merge commit:
"1c757b8a2c1160ad53421b7b8bd7f74b205c4b89", automatic backport requires
rebase workflow
The below text is copied from the body of the original PR.
---
fixes#17097 Consul version of each nodes in UI nodes section
@jkirschner-hashicorp @huikang @team @Maintainers
Updated consul version in the request to register consul.
Added this as Node MetaData.
Fetching this new metadata in UI
<img width="1512" alt="Screenshot 2023-06-15 at 4 21 33 PM"
src="https://github.com/hashicorp/consul/assets/3139634/94f7cf6b-701f-4230-b9f7-d8c4342d0737">
Also made this backward compatible and tested.
Backward compatible in this context means - If consul binary with above
PR changes is deployed to one of node, and if UI is run from this node,
then the version of not only current (upgraded) node is displayed in UI
, but also of older nodes given that they are consul servers only.
For older (non-server or client) nodes the version is not added in
NodeMeta Data and hence the version will not be displayed for them.
If a old node is consul server, the version will be displayed. As the
endpoint - "v1/internal/ui/nodes?dc=dc1" was already returning version
in service meta. This is made use of in current UI changes.
<img width="1480" alt="Screenshot 2023-06-16 at 6 58 32 PM"
src="https://github.com/hashicorp/consul/assets/3139634/257942f4-fbed-437d-a492-37849d2bec4c">
---
<details>
<summary> Overview of commits </summary>
- 931fdfc7ecdc26bb7cc20b698c5e14c1b65fcc6e -
b3e2ec1ccaca3832a088ffcac54257fa6653c6c1 -
8d0e9a54907039c09330c6cd7b9e761566af6856 -
04e5d88cca37821f6667be381c16aaa5958b5c92 -
28286a2e98f8cd66ef8593c2e2893b4db6080417 -
43e50ad38207952a9c4d04d45d08b6b8f71b31fe -
0cf1b7077cdf255596254d9dc1624a269c42b94d -
27f34ce1c2973591f75b1e38a81ccbe7cee6cee3 -
2ac76d62b8cbae76b1a903021aebb9b865e29d6e -
3d618df9ef1d10dd5056c8b1ed865839c553a0e0 -
1c757b8a2c1160ad53421b7b8bd7f74b205c4b89 -
23ce82b4cee8f74dd634dbe145313e9a56c0077d -
4dc1c9b4c5aafdb8883ef977dfa9b39da138b6cb -
85a12a92528bfa267a039a9bb258170be914abf7 -
25d30a3fa980d130a30d445d26d47ef2356cb553 -
7f1d6192dce3352e92307175848b89f91e728c24 -
5174cbff84b0795d4cb36eb8980d0d5336091ac9
</details>
---------
Co-authored-by: Vijay Srinivas <vijayraghav22@gmail.com>
Co-authored-by: John Murret <john.murret@hashicorp.com>
Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com>
## Backport
This PR is auto-generated from #18062 to be assessed for backporting due
to the inclusion of the label backport/1.16.
The below text is copied from the body of the original PR.
---
### Description
<!-- Please describe why you're making this change, in plain English.
-->
- Currently the jwt-auth filter doesn't take into account the service
identity when validating jwt-auth, it only takes into account the path
and jwt provider during validation. This causes issues when multiple
source intentions restrict access to an endpoint with different JWT
providers.
- To fix these issues, rather than use the JWT auth filter for
validation, we use it in metadata mode and allow it to forward the
successful validated JWT token payload to the RBAC filter which will
make the decisions.
This PR ensures requests with and without JWT tokens successfully go
through the jwt-authn filter. The filter however only forwards the data
for successful/valid tokens. On the RBAC filter level, we check the
payload for claims and token issuer + existing rbac rules.
### Testing & Reproduction steps
<!--
* In the case of bugs, describe how to replicate
* If any manual tests were done, document the steps and the conditions
to replicate
* Call out any important/ relevant unit tests, e2e tests or integration
tests you have added or are adding
-->
- This test covers a multi level jwt requirements (requirements at top
level and permissions level). It also assumes you have envoy running,
you have a redis and a sidecar proxy service registered, and have a way
to generate jwks with jwt. I mostly use:
https://www.scottbrady91.com/tools/jwt for this.
- first write your proxy defaults
```
Kind = "proxy-defaults"
name = "global"
config {
protocol = "http"
}
```
- Create two providers
```
Kind = "jwt-provider"
Name = "auth0"
Issuer = "https://ronald.local"
JSONWebKeySet = {
Local = {
JWKS = "eyJrZXlzIjog....."
}
}
```
```
Kind = "jwt-provider"
Name = "okta"
Issuer = "https://ronald.local"
JSONWebKeySet = {
Local = {
JWKS = "eyJrZXlzIjogW3...."
}
}
```
- add a service intention
```
Kind = "service-intentions"
Name = "redis"
JWT = {
Providers = [
{
Name = "okta"
},
]
}
Sources = [
{
Name = "*"
Permissions = [{
Action = "allow"
HTTP = {
PathPrefix = "/workspace"
}
JWT = {
Providers = [
{
Name = "okta"
VerifyClaims = [
{
Path = ["aud"]
Value = "my_client_app"
},
{
Path = ["sub"]
Value = "5be86359073c434bad2da3932222dabe"
}
]
},
]
}
},
{
Action = "allow"
HTTP = {
PathPrefix = "/"
}
JWT = {
Providers = [
{
Name = "auth0"
},
]
}
}]
}
]
```
- generate 3 jwt tokens: 1 from auth0 jwks, 1 from okta jwks with
different claims than `/workspace` expects and 1 with correct claims
- connect to your envoy (change service and address as needed) to view
logs and potential errors. You can add: `-- --log-level debug` to see
what data is being forwarded
```
consul connect envoy -sidecar-for redis1 -grpc-addr 127.0.0.1:8502
```
- Make the following requests:
```
curl -s -H "Authorization: Bearer $Auth0_TOKEN" --insecure --cert leaf.cert --key leaf.key --cacert connect-ca.pem https://localhost:20000/workspace -v
RBAC filter denied
curl -s -H "Authorization: Bearer $Okta_TOKEN_with_wrong_claims" --insecure --cert leaf.cert --key leaf.key --cacert connect-ca.pem https://localhost:20000/workspace -v
RBAC filter denied
curl -s -H "Authorization: Bearer $Okta_TOKEN_with_correct_claims" --insecure --cert leaf.cert --key leaf.key --cacert connect-ca.pem https://localhost:20000/workspace -v
Successful request
```
### TODO
* [x] Update test coverage
* [ ] update integration tests (follow-up PR)
* [x] appropriate backport labels added
---
<details>
<summary> Overview of commits </summary>
- 70536f5a38507d7468f62d00dd93a6968a3d9cf3
</details>
Co-authored-by: Ronald Ekambi <ronekambi@gmail.com>
* backport of commit 4034bb2b3eba81ea13bf6d3a62d27094d96ffc24
* backport of commit 9c4c3c50f07d4072bb981c16cf993118fd7f6f1d
* backport of commit 7282078993aa51915afa801bdabded0f78397cb5
---------
Co-authored-by: Tom Davies <thomas.23.davies@bt.com>
* backport of commit afa1f42cc719b13074f2f286202d8f21b8000753
* backport of commit e0970025d4c2e2702af30e642b37dd5e32561756
* backport of commit 2f2aad545b1ebcae22bb481b57115a679eb539e5
* backport of commit 4a5c9c181f50343911cd30fbb0f0475e473a2c7b
---------
Co-authored-by: Ranjandas <thejranjan@gmail.com>
Co-authored-by: Chris S. Kim <kisunji92@gmail.com>
* backport of commit 93ccfe4c1195ba0ab2d12443f25d9cf29e9e4f0c
* Ensure RSA keys are at least 2048 bits in length (#17911)
* Ensure RSA keys are at least 2048 bits in length
* Add changelog
* update key length check for FIPS compliance
* Fix no new variables error and failing to return when error exists from
validating
* clean up code for better readability
* actually return value
---------
Co-authored-by: jm96441n <john.maguire@hashicorp.com>
* backport of commit 72308dd9f2a17db4c7c8ea7eabb55db3adadaa91
* backport of commit bf5d1ec2ec68fd7428027244a094baeee49d4396
* backport of commit 4ca2e4be490655948f7480cbc182c6cca7cec0a9
---------
Co-authored-by: Ronald Ekambi <ronekambi@gmail.com>
* backport of commit 391db7e58b501b3ed7561fec352f2f3f5004a29f
* backport of commit f204d5b52ab80836128882a65d7d7c5e53b2fa3d
---------
Co-authored-by: Chris Thain <chris.m.thain@gmail.com>
* backport of commit d77048f1ea7136af9f627182bc79126f9472a060
* backport of commit 301de5980e7fcc8d68d16e6cdcbd5b2afea73711
* backport of commit 436bcda5ec58abbc5a65f6d244a1e1ec26beb46c
---------
Co-authored-by: Ronald Ekambi <ronekambi@gmail.com>
* backport of commit 97c779b5a2308a05fde93247209fa6e9cd3fc310
* backport of commit dd56a6800bebc54dabd7883fddc22b25ca2bdb92
---------
Co-authored-by: Michael Zalimeni <michael.zalimeni@hashicorp.com>
* backport of commit ad6652a47b5c5f9e1b8aed673922b918f56dbfcc
* backport of commit 7a1e99b6b774257ef6e293c2d47375e3b2332b9a
---------
Co-authored-by: Matt Keeler <mjkeeler7@gmail.com>
* backport of commit 131d234bdab165d96601b8064c49ce17ee5f141a
* backport of commit 1adc48734d9347599c8a694d427c6b26e5a748a1
---------
Co-authored-by: Chris Thain <chris.m.thain@gmail.com>
* backport of commit fb2f3b61004d9ef2296b51306ddbf5b6d72679ed
* backport of commit 178abb8495ba4bb35c29a835965e3f244a385865
* backport of commit 77b399877413c6e65669659deb8962c2cc5b52f6
* backport of commit a245b326ac030f7ef3292e7bbdb58e0b2850a12f
---------
Co-authored-by: Andrew Stucki <andrew.stucki@hashicorp.com>
* backport of commit 63e87434721824902d1bd8c7ef72456615020180
* backport of commit 896f11789b89062c745433a17dc37da5b56185fd
---------
Co-authored-by: Andrew Stucki <andrew.stucki@hashicorp.com>
* backport of commit d77784ba51fd6a5d598ea2b87cb6e36e0fed8e72
* backport of commit f5a557dd7a5995094b3af96f1c522d49acfe795b
* backport of commit 1d782d63c437ab16e30d5bd00a6b8c3cbad08845
---------
Co-authored-by: Ronald Ekambi <ronekambi@gmail.com>
Ensure that the embedded api struct is properly parsed when
deserializing config containing a set ResourceFilter.Services field.
Also enhance existing integration test to guard against bugs and
exercise this field.
TLDR with many modules the versions included in each diverged quite a bit. Attempting to use Go Workspaces produces a bunch of errors.
This commit:
1. Fixes envoy-library-references.sh to work again
2. Ensures we are pulling in go-control-plane@v0.11.0 everywhere (previously it was at that version in some modules and others were much older)
3. Remove one usage of golang/protobuf that caused us to have a direct dependency on it.
4. Remove deprecated usage of the Endpoint field in the grpc resolver.Target struct. The current version of grpc (v1.55.0) has removed that field and recommended replacement with URL.Opaque and calls to the Endpoint() func when needing to consume the previous field.
4. `go work init <all the paths to go.mod files>` && `go work sync`. This syncrhonized versions of dependencies from the main workspace/root module to all submodules
5. Updated .gitignore to ignore the go.work and go.work.sum files. This seems to be standard practice at the moment.
6. Update doc comments in protoc-gen-consul-rate-limit to be go fmt compatible
7. Upgraded makefile infra to perform linting, testing and go mod tidy on all modules in a flexible manner.
8. Updated linter rules to prevent usage of golang/protobuf
9. Updated a leader peering test to account for an extra colon in a grpc error message.
Dhia Ayachi