7d48383041
Avoid panic on concurrent writes to cached service config map ( #10647 )
...
If multiple instances of a service are co-located on the same node then
their proxies will all share a cache entry for their resolved service
configuration. This is because the cache key contains the name of the
watched service but does not take into account the ID of the watching
proxies.
This means that there will be multiple agent service manager watches
that can wake up on the same cache update. These watchers then
concurrently modify the value in the cache when merging the resolved
config into the local proxy definitions.
To avoid this concurrent map write we will only delete the key from
opaque config in the local proxy definition after the merge, rather
than from the cached value before the merge.
2021-07-20 10:09:29 -06:00
85c36bd229
website: Fix circular redirect with TLS on existing cluster
...
Fix an issue where /docs/k8s/operations/tls-on-existing-cluster would
never load when navigating directly to the URL because of a circular
redirect.
2021-07-20 08:41:43 -07:00
aa97ed5ac6
auto-updated agent/uiserver/bindata_assetfs.go from commit 1eb7a83ee
2021-07-20 15:15:10 +00:00
116a255084
ui: Add tests for topology metrics stats ( #10600 )
2021-07-20 11:09:15 -04:00
9a84fe7864
docs: Add intentions to ACL System docs ( #10323 )
...
Adds mention of `intentions` rules to ACL System and ACL Rules pages.
Resolves #9790
2021-07-19 15:31:41 -07:00
441a6c9969
Add DNS recursor strategy option ( #10611 )
...
This change adds a new `dns_config.recursor_strategy` option which
controls how Consul queries DNS resolvers listed in the `recursors`
config option. The supported options are `sequential` (default), and
`random`.
Closes #8807
Co-authored-by: Blake Covarrubias <blake@covarrubi.as>
Co-authored-by: Priyanka Sengupta <psengupta@flatiron.com>
2021-07-19 15:22:51 -07:00
4d2bc76d62
docs: Fix spelling errors across website
2021-07-19 14:29:54 -07:00
901a5cdd8c
Merge pull request #10396 from hashicorp/dnephin/fix-more-data-races
...
Fix some data races
2021-07-16 18:21:58 -04:00
23dfb8e9ad
Merge pull request #10009 from hashicorp/dnephin/trim-dns-response-with-edns
...
dns: properly trim response when EDNS is used
2021-07-16 18:09:25 -04:00
db29c51cd2
acl: use SetHash consistently in testPolicyForID
...
A previous commit used SetHash on two of the cases to fix a data race. This commit applies
that change to all cases. Using SetHash in this test helper should ensure that the
test helper behaves closer to production.
2021-07-16 17:59:56 -04:00
0cb479f782
Merge pull request #10353 from hashicorp/dnephin/prune-build-scripts-1
...
Remove a few unused things from build-support
2021-07-16 14:27:00 -04:00
63772f7ac4
dns: improve naming of error to match DNS terminology
...
Co-authored-by: Blake Covarrubias <blake@covarrubi.as>
2021-07-16 12:40:24 -04:00
079decdabd
fix truncate when NS is set
...
Also: fix test to catch the issue
2021-07-16 12:40:11 -04:00
521c423075
acls: Show `AuthMethodNamespace` when reading/listing ACL token meta ( #10598 )
2021-07-15 10:38:52 -07:00
b4ab87111c
Merge pull request #10567 from hashicorp/dnephin/config-unexport-build
...
config: unexport the remaining builder methods
2021-07-15 12:05:19 -04:00
a942a2e025
Merge pull request #10621 from hashicorp/vuln/validate-sans
2021-07-15 09:43:55 -06:00
cfc31f957b
Add changelog entry
2021-07-15 09:27:46 -06:00
4c78825f0c
Merge pull request #10617 from hashicorp/dnephin/config-add-missing-docs
...
docs: add config options that were missing
2021-07-15 11:23:32 -04:00
f286ea0922
Fix godoc comment
...
Co-authored-by: Freddy <freddygv@users.noreply.github.com>
2021-07-15 11:22:46 -04:00
b362ce092e
Merge pull request #10618 from hashicorp/dnephin/docs-add-deprecation-version-grpc-port
...
docs: add deprecation version for ports.grpc settings
2021-07-15 11:14:51 -04:00
e018d8a10b
xds: ensure single L7 deny intention with default deny policy does not result in allow action (CVE-2021-36213) ( #10619 )
2021-07-15 10:09:00 -05:00
6bf7c98227
auto-updated agent/uiserver/bindata_assetfs.go from commit 0762da3a6
2021-07-15 11:23:49 +00:00
ffbe54971f
ui: [BUGFIX] Ensure we use the ns query param name when requesting permissions ( #10608 )
...
Previously when namespaces were enabled, we weren't requesting permission for the actively selected namespace, and instead always checking the permissions for the default namespace.
This commit ensures we request permissions for the actively selected namespace.
2021-07-15 12:19:07 +01:00
3a1afd8f57
acl: fix error type into a string type for serialization issue
...
acl_endpoint_test.go:507:
Error Trace: acl_endpoint_test.go:507
retry.go:148
retry.go:149
retry.go:103
acl_endpoint_test.go:504
Error: Received unexpected error:
codec.decoder: decodeValue: Cannot decode non-nil codec value into nil error (1 methods)
Test: TestACLEndpoint_ReplicationStatus
2021-07-15 11:31:44 +02:00
b6b42c34dc
Add TODOs about partition handling
2021-07-14 22:21:55 -06:00
3d4fa44c22
Update golden files
2021-07-14 22:21:55 -06:00
a7de87e95b
Validate SANs for passthrough clusters and failovers
2021-07-14 22:21:55 -06:00
a6f7d806f6
Update golden files to account for SAN validation
2021-07-14 22:21:55 -06:00
3f11449363
Validate Subject Alternative Name for upstreams
...
These changes ensure that the identity of services dialed is
cryptographically verified.
For all upstreams we validate against SPIFFE IDs in the format used by
Consul's service mesh:
spiffe://<trust-domain>/ns/<namespace>/dc/<datacenter>/svc/<service>
2021-07-14 22:20:27 -06:00
27871498f0
Fix a data race in TestACLResolver_Client
...
By setting the hash when we create the policy.
```
WARNING: DATA RACE
Read at 0x00c0028b4b10 by goroutine 1182:
github.com/hashicorp/consul/agent/structs.(*ACLPolicy).SetHash()
/home/daniel/pers/code/consul/agent/structs/acl.go:701 +0x40d
github.com/hashicorp/consul/agent/structs.ACLPolicies.resolveWithCache()
/home/daniel/pers/code/consul/agent/structs/acl.go:779 +0xfe
github.com/hashicorp/consul/agent/structs.ACLPolicies.Compile()
/home/daniel/pers/code/consul/agent/structs/acl.go:809 +0xf1
github.com/hashicorp/consul/agent/consul.(*ACLResolver).ResolveTokenToIdentityAndAuthorizer()
/home/daniel/pers/code/consul/agent/consul/acl.go:1226 +0x6ef
github.com/hashicorp/consul/agent/consul.resolveTokenAsync()
/home/daniel/pers/code/consul/agent/consul/acl_test.go:66 +0x5c
Previous write at 0x00c0028b4b10 by goroutine 1509:
github.com/hashicorp/consul/agent/structs.(*ACLPolicy).SetHash()
/home/daniel/pers/code/consul/agent/structs/acl.go:730 +0x3a8
github.com/hashicorp/consul/agent/structs.ACLPolicies.resolveWithCache()
/home/daniel/pers/code/consul/agent/structs/acl.go:779 +0xfe
github.com/hashicorp/consul/agent/structs.ACLPolicies.Compile()
/home/daniel/pers/code/consul/agent/structs/acl.go:809 +0xf1
github.com/hashicorp/consul/agent/consul.(*ACLResolver).ResolveTokenToIdentityAndAuthorizer()
/home/daniel/pers/code/consul/agent/consul/acl.go:1226 +0x6ef
github.com/hashicorp/consul/agent/consul.resolveTokenAsync()
/home/daniel/pers/code/consul/agent/consul/acl_test.go:66 +0x5c
Goroutine 1182 (running) created at:
github.com/hashicorp/consul/agent/consul.TestACLResolver_Client.func4()
/home/daniel/pers/code/consul/agent/consul/acl_test.go:1669 +0x459
testing.tRunner()
/usr/lib/go/src/testing/testing.go:1193 +0x202
Goroutine 1509 (running) created at:
github.com/hashicorp/consul/agent/consul.TestACLResolver_Client.func4()
/home/daniel/pers/code/consul/agent/consul/acl_test.go:1668 +0x415
testing.tRunner()
/usr/lib/go/src/testing/testing.go:1193 +0x202
```
2021-07-14 18:58:16 -04:00
291315e39f
Update serf
...
To pick up data race fixes
2021-07-14 18:58:16 -04:00
c3c8058fd7
agent: remove deprecated call in a test
2021-07-14 18:58:16 -04:00
9d471269d8
agent: fix a data race in a test
...
The test was modifying a pointer to a struct that had been passed to
another goroutine. Instead create a new struct to modify.
```
WARNING: DATA RACE
Write at 0x00c01407c3c0 by goroutine 832:
github.com/hashicorp/consul/agent.TestServiceManager_PersistService_API()
/home/daniel/pers/code/consul/agent/service_manager_test.go:446 +0x1d86
testing.tRunner()
/usr/lib/go/src/testing/testing.go:1193 +0x202
Previous read at 0x00c01407c3c0 by goroutine 938:
reflect.typedmemmove()
/usr/lib/go/src/runtime/mbarrier.go:177 +0x0
reflect.Value.Set()
/usr/lib/go/src/reflect/value.go:1569 +0x13b
github.com/mitchellh/copystructure.(*walker).Primitive()
/home/daniel/go/pkg/mod/github.com/mitchellh/copystructure@v1.0.0/copystructure.go:289 +0x190
github.com/mitchellh/reflectwalk.walkPrimitive()
/home/daniel/go/pkg/mod/github.com/mitchellh/reflectwalk@v1.0.1/reflectwalk.go:252 +0x31b
github.com/mitchellh/reflectwalk.walk()
/home/daniel/go/pkg/mod/github.com/mitchellh/reflectwalk@v1.0.1/reflectwalk.go:179 +0x24d
github.com/mitchellh/reflectwalk.walkStruct()
/home/daniel/go/pkg/mod/github.com/mitchellh/reflectwalk@v1.0.1/reflectwalk.go:386 +0x4ec
github.com/mitchellh/reflectwalk.walk()
/home/daniel/go/pkg/mod/github.com/mitchellh/reflectwalk@v1.0.1/reflectwalk.go:188 +0x656
github.com/mitchellh/reflectwalk.walkStruct()
/home/daniel/go/pkg/mod/github.com/mitchellh/reflectwalk@v1.0.1/reflectwalk.go:386 +0x4ec
github.com/mitchellh/reflectwalk.walk()
/home/daniel/go/pkg/mod/github.com/mitchellh/reflectwalk@v1.0.1/reflectwalk.go:188 +0x656
github.com/mitchellh/reflectwalk.Walk()
/home/daniel/go/pkg/mod/github.com/mitchellh/reflectwalk@v1.0.1/reflectwalk.go:92 +0x164
github.com/mitchellh/copystructure.Config.Copy()
/home/daniel/go/pkg/mod/github.com/mitchellh/copystructure@v1.0.0/copystructure.go:69 +0xe7
github.com/mitchellh/copystructure.Copy()
/home/daniel/go/pkg/mod/github.com/mitchellh/copystructure@v1.0.0/copystructure.go:13 +0x84
github.com/hashicorp/consul/agent.mergeServiceConfig()
/home/daniel/pers/code/consul/agent/service_manager.go:362 +0x56
github.com/hashicorp/consul/agent.(*serviceConfigWatch).handleUpdate()
/home/daniel/pers/code/consul/agent/service_manager.go:279 +0x250
github.com/hashicorp/consul/agent.(*serviceConfigWatch).runWatch()
/home/daniel/pers/code/consul/agent/service_manager.go:246 +0x2d4
Goroutine 832 (running) created at:
testing.(*T).Run()
/usr/lib/go/src/testing/testing.go:1238 +0x5d7
testing.runTests.func1()
/usr/lib/go/src/testing/testing.go:1511 +0xa6
testing.tRunner()
/usr/lib/go/src/testing/testing.go:1193 +0x202
testing.runTests()
/usr/lib/go/src/testing/testing.go:1509 +0x612
testing.(*M).Run()
/usr/lib/go/src/testing/testing.go:1417 +0x3b3
main.main()
_testmain.go:1181 +0x236
Goroutine 938 (running) created at:
github.com/hashicorp/consul/agent.(*serviceConfigWatch).start()
/home/daniel/pers/code/consul/agent/service_manager.go:223 +0x4e4
github.com/hashicorp/consul/agent.(*ServiceManager).AddService()
/home/daniel/pers/code/consul/agent/service_manager.go:98 +0x344
github.com/hashicorp/consul/agent.(*Agent).addServiceLocked()
/home/daniel/pers/code/consul/agent/agent.go:1942 +0x2e4
github.com/hashicorp/consul/agent.(*Agent).AddService()
/home/daniel/pers/code/consul/agent/agent.go:1929 +0x337
github.com/hashicorp/consul/agent.TestServiceManager_PersistService_API()
/home/daniel/pers/code/consul/agent/service_manager_test.go:400 +0x17c4
testing.tRunner()
/usr/lib/go/src/testing/testing.go:1193 +0x202
```
2021-07-14 18:58:16 -04:00
6703787740
agent: fix a data race in DNS tests
...
The dnsConfig pulled from the atomic.Value is a pointer, so modifying it in place
creates a data race. Use the exported ReloadConfig interface instead.
2021-07-14 18:58:16 -04:00
2946e42a9e
agent: fix two data race in agent tests
...
The LogOutput io.Writer used by TestAgent must allow concurrent reads and writes, and a
bytes.Buffer does not allow this. The bytes.Buffer must be wrapped with a lock to make this safe.
2021-07-14 18:58:16 -04:00
ff26294d63
consul: fix data race in leader CA tests
...
Some global variables are patched to shorter values in these tests. But the goroutines that read
them can outlive the test because nothing waited for them to exit.
This commit adds a Wait() method to the routine manager, so that tests can wait for the goroutines
to exit. This prevents the data race because the 'reset to original value' can happen
after all other goroutines have stopped.
2021-07-14 18:58:15 -04:00
5ae7c6a490
add changelog entry
2021-07-14 17:50:00 -04:00
edd755b7ab
dns: correct rcode for qtype not supported
...
A previous commit started using QueryRefuced, but that is not correct. QueryRefuced refers to
the OpCode, not the query type.
Instead use errNoAnswer because we have no records for that query type.
2021-07-14 17:48:50 -04:00
48171c43f4
Check response len do not exceed max Buffer size
2021-07-14 17:15:34 -04:00
8fcac3cef6
add missing test for truncate
2021-07-14 17:15:34 -04:00
b4abf8b0ec
dns: remove network parameter from two funcs
...
Now that trimDNSResponse is handled by the caller we don't need to pass this value
around. We can remove it from both the serviceLookup struct, and two functions.
2021-07-14 17:15:34 -04:00
4712e24749
dns: trim response immediately before the write
...
Previously the response was being trimmed before adding the EDNS values, which could cause it to exceed
the max size.
2021-07-14 17:15:34 -04:00
a9e9c6c23e
dns: handle errors from dispatch
2021-07-14 17:15:34 -04:00
6cf9ecc1c9
dns: error response from dispatch
...
So that dispatch can communicate status back to the caller.
2021-07-14 17:15:34 -04:00
9298cfe0f6
dns: refactor dispatch to use an explicit return in each case
...
In preparation for changing the return value, so that SOA, eDNS trimming and 'not found'
errors can be handled in a single place.
2021-07-14 17:15:34 -04:00
b09aa1e3c6
dns: small refactor to setEDNS to return early
...
Using a guard clause instead of a long nested if.
The diff is best viewed with whitespace turned off.
2021-07-14 17:15:34 -04:00
f1bc7bd49a
dns: remove unused method
...
It was added in 5934f803bfb54c1ceeeb6518398f1b82a726459f but it was never used.
2021-07-14 17:15:34 -04:00
e3d781d99c
dns: remove unnecessary function wrapping
...
The dispatch function was called from a single place and did nothing but add a default value.
Removing it makes code easier to trace by removing an unnecessary hop.
2021-07-14 17:15:33 -04:00
ce6846fba4
docs: add deprecation version for ports.grpc settings
2021-07-14 17:06:13 -04:00
7fff096d8d
Merge pull request #10568 from hashicorp/dnephin/contrib-docs-config
...
contrib: add first draft of agent configuration docs
2021-07-14 16:54:36 -04:00
Freddy