Commit Graph

2868 Commits

Author SHA1 Message Date
Daniel Nephin 979749d86e state: do not delete from inside an iteration
Deleting from memdb inside an interation can cause a panic from Iterator.Next. This
case is technically safe (for now) because the iterator is using the root radix tree
not a modified one.

However this could break at any time if someone adds an insert or delete to the coordinates table
before this place in the function.

It also sets a bad example, because generally deletes in an interator are not safe. So this
commit uses the pattern we have in other places to move the deletes out of the iteration.
2021-01-19 17:00:07 -05:00
Matt Keeler 2d7a4073e3
Merge pull request #9570 from hashicorp/bugfix/9498 2021-01-19 16:30:04 -05:00
Matt Keeler 2d2ce1fb0c
Ensure that CA initialization does not block leader election.
After fixing that bug I uncovered a couple more:

Fix an issue where we might try to cross sign a cert when we never had a valid root.
Fix a potential issue where reconfiguring the CA could cause either the Vault or AWS PCA CA providers to delete resources that are still required by the new incarnation of the CA.
2021-01-19 15:27:48 -05:00
hashicorp-ci 4b3361fb03 auto-updated agent/uiserver/bindata_assetfs.go from commit be694366a 2021-01-19 15:47:02 +00:00
hashicorp-ci 88ab4cd159 auto-updated agent/uiserver/bindata_assetfs.go from commit 41a4a9f4f 2021-01-19 15:29:55 +00:00
Daniel Nephin 52a1d78e39 state: add a regression test for state store schema
To allow the index to be refactored without accidental changes.

To update the expected value run: 'go test ./agent/consul/state -update'
2021-01-15 18:49:55 -05:00
Daniel Nephin aa21c1ea04 state: reduce interface for Enterprise schema
Using withEnterpriseSchema() we can apply any enterprise schema changes
with a single shim, removing the need to duplicate all of the table
definitions.

Also move all the catalog schemas to a new file to shrink catalog.go a bit.
2021-01-15 18:49:55 -05:00
Daniel Nephin 45c0a71e7e
Merge pull request #8696 from hashicorp/dnephin/fix-load-limits
agent/consul: make Client/Server config reloading more obvious
2021-01-14 17:40:42 -05:00
Daniel Nephin 27a59f86d0
Merge pull request #9436 from hashicorp/dnephin/fix-service-health-req-cache-key
structs: fix caching of ServiceSpecificRequest when ingress=true
2021-01-14 17:26:25 -05:00
Daniel Nephin e8427a48ab agent/consuk: Rename RPCRate -> RPCRateLimit
so that the field name is consistent across config structs.
2021-01-14 17:26:00 -05:00
Daniel Nephin e5320c2db6 agent/consul: make Client/Server config reloading more obvious
I believe this commit also fixes a bug. Previously RPCMaxConnsPerClient was not being re-read from the RuntimeConfig, so passing it to Server.ReloadConfig was never changing the value.

Also improve the test runtime by not doing a lot of unnecessary work.
2021-01-14 17:21:10 -05:00
Daniel Nephin f2b504873a
Merge pull request #9460 from hashicorp/dnephin/fix-data-races
Fix a couple data races in tests
2021-01-14 17:07:01 -05:00
Daniel Nephin ef9d44fdfe structs: fix caching of ServiceSpecificRequest when ingress=true
The field was not being included in the cache info key. This would result in a DNS request for
web.service.consul returning the same result as web.ingress.consul, when those results should
not be the same.
2021-01-14 17:01:40 -05:00
hashicorp-ci 66ec8e5e1f auto-updated agent/uiserver/bindata_assetfs.go from commit 1e30503ec 2021-01-13 09:47:00 +00:00
kevinkengne 18a12d6517
add completeness test for types with CacheInfo method (#9480)
include all fields when fuzzing in tests
split tests by struct type

Ensure the new value for the field is different

fuzzer.Fuzz could produce the same value again in some cases.

Use a custom fuzz function for QueryOptions. That type is an embedded struct in the request types
but only one of the fields is important to include in the cache key.

Move enterpriseMetaField to an oss file so that we can change it in enterprise.
2021-01-12 19:45:46 -05:00
Chris Piraino baad708929
Fix bug in usage metrics when multiple service instances are changed in a single transaction (#9440)
* Fix bug in usage metrics that caused a negative count to occur

There were a couple of instances were usage metrics would do the wrong
thing and result in incorrect counts, causing the count to attempt to
decrement below zero and return an error. The usage metrics did not
account for various places where a single transaction could
delete/update/add multiple service instances at once.

We also remove the error when attempting to decrement below zero, and
instead just make sure we do not accidentally underflow the unsigned
integer. This is a more graceful failure than returning an error and not
allowing a transaction to commit.

* Add changelog
2021-01-12 15:31:47 -06:00
hashicorp-ci 5b92255cf5 auto-updated agent/uiserver/bindata_assetfs.go from commit b86eea4fb 2021-01-12 14:57:52 +00:00
Daniel Nephin 51464763f4 config: remove Builder receiver from funcs that dont use it
This change allows us to re-use these functions in other places without the Builder, and makes it
more explicit about which functions can warn/error and which can not.
2021-01-11 17:41:58 -05:00
Daniel Nephin 78b02663e9 config: Use golden for TestRuntimeConfig_Sanitize
A golden file makes the expected value easier to work with. This change also
removes a number of shims for enterprise and replaces them with a single one
for the golden filename.
2021-01-11 14:34:03 -05:00
Pierre Souchay 70d2da7582
Display a warning when rpc.enable_streaming = true is set on a client (#9530)
* Display a warning when rpc.enable_streaming = true is set on a client

This option has no effect when running as an agent

* Added warning when server starts with use_streaming_backend but without rpc.enable_streaming

* Added unit test
2021-01-08 15:23:23 -05:00
Chris Piraino 2eac571276
Log replication warnings when no error suppression is defined (#9320)
* Log replication warnings when no error suppression is defined

* Add changelog file
2021-01-08 14:03:06 -06:00
Daniel Nephin 6094b73d89
Merge pull request #9250 from hashicorp/dnephin/config-tags-2
config: remove unnecessary json/hcl struct tags
2021-01-08 14:08:34 -05:00
Daniel Nephin d5bdc2f539
Merge pull request #9249 from hashicorp/dnephin/config-tags
config: use fields to detect enterprise-only settings
2021-01-07 19:49:29 -05:00
Daniel Nephin f3b30a7ed6 config: remove unnecessary json/hcl struct tags
Using find/replace with regex

        json:"[^"]+"
        hcl:"[^"]+"
2021-01-07 19:42:56 -05:00
Daniel Nephin ee00d7668c config: Use config fields to warn about enterprise settings
It is no safe to assumes that the mapstructure keys will contain all the keys because some config can be specified
with command line flags or literals.

This change allows us to remove the json marshal/unmarshal cycle for command line flags, which will allow
us to remove all of the hcl/json struct tags on config fields.
2021-01-07 19:31:39 -05:00
Daniel Nephin 0679a23921
Merge pull request #9251 from hashicorp/dnephin/config-cleanup-flags-and-opts
config: move testing shims and cleanup some defaults
2021-01-07 19:11:22 -05:00
Daniel Nephin 942ffa7e02 config: use LiteralConfig for flag values 2021-01-07 18:57:53 -05:00
Daniel Nephin 2d0f33591e config: remove t.Parallel and some unused fields from tests
Use go-cmp for better diffs in tests
2021-01-07 18:57:52 -05:00
Daniel Nephin 31b1addd9e structs: add tests for String() methods
To show that printing one of these IDs works properly now that the String() method
receiver is no longer a pointer.
2021-01-07 18:47:38 -05:00
Daniel Nephin 45f0afcbf4 structs: Fix printing of IDs
These types are used as values (not pointers) in other structs. Using a pointer receiver causes
problems when the value is printed. fmt will not call the String method if it is passed a value
and the String method has a pointer receiver. By using a value receiver the correct string is printed.

Also remove some unused methods.
2021-01-07 18:47:38 -05:00
Daniel Nephin bcfb444a77 Remove an unnecessary else 2021-01-07 18:13:49 -05:00
Daniel Nephin f6543b1651 xds: remove Server.Initialize
Requiring a call to initialize to set a single field is not really substantially different
from having to set that field to a value.
2021-01-07 18:13:48 -05:00
Daniel Nephin bbf1a116f6 xds: Fix data race
TestEnvoy.Close used e.stream.recvCh == nil to indicate the channel had already
been closed, so that TestEnvoy.Close can be called multiple times. The recvCh
was not protected by a lock, so setting it to nil caused a data race with any
goroutine trying to read from the channel.

Instead set the stream to nil. The stream is guarded by a lock, so it does not race.

This change allows us to test the agent/xds package using -race.
2021-01-07 18:13:48 -05:00
Daniel Nephin de226f26e4 xds: Pass in logger
small cleanup in tests
2021-01-07 18:13:48 -05:00
hashicorp-ci 7c5ca27fdd auto-updated agent/uiserver/bindata_assetfs.go from commit e893ba7ea 2021-01-07 19:09:58 +00:00
Daniel Nephin 27c38bfebb
Merge pull request #9213 from hashicorp/dnephin/resolve-tokens-take-2
acl: Remove some unused things and document delegate method
2021-01-06 18:51:51 -05:00
Pierre Souchay c43888c064 Added testing of GRPC with TLS combinations
This ensures that https://github.com/hashicorp/consul/issues/9474 will
not reproduce.
2021-01-06 22:20:23 +01:00
Pierre Souchay 542852786c [Streaming][bugfix] handle TLS signalisation when TLS is disabled on client side
Tnis is an alternative to https://github.com/hashicorp/consul/pull/9494
2021-01-06 17:24:58 +01:00
R.B. Boyer db62541676
acl: use the presence of a management policy in the state store as a sign that we already migrated to v2 acls (#9505)
This way we only have to wait for the serf barrier to pass once before
we can upgrade to v2 acls. Without this patch every restart needs to
re-compute the change, and potentially if a stray older node joins after
a migration it might regress back to v1 mode which would be problematic.
2021-01-05 17:04:27 -06:00
hashicorp-ci 8fc77877be auto-updated agent/uiserver/bindata_assetfs.go from commit 1304dc882 2021-01-05 17:47:53 +00:00
Daniel Nephin 3140c0a343
Merge pull request #9067 from naemono/6074-allow-config-MaxHeaderBytes
Adds option to configure HTTP Server's MaxHeaderBytes
2021-01-05 12:28:27 -05:00
hashicorp-ci e64b90c604 auto-updated agent/uiserver/bindata_assetfs.go from commit a42e844cc 2021-01-05 17:09:19 +00:00
hashicorp-ci 988609fb61 auto-updated agent/uiserver/bindata_assetfs.go from commit 17438020f 2021-01-05 10:11:12 +00:00
Michael Montgomery 5a70c2c7e8 Remove unneeded test 2021-01-04 19:47:13 -06:00
Matt Keeler 3a79b559f9
Special case the error returned when we have a Raft leader but are not tracking it in the ServerLookup (#9487)
This can happen when one other node in the cluster such as a client is unable to communicate with the leader server and sees it as failed. When that happens its failing status eventually gets propagated to the other servers in the cluster and eventually this can result in RPCs returning “No cluster leader” error.

That error is misleading and unhelpful for determing the root cause of the issue as its not raft stability but rather and client -> server networking issue. Therefore this commit will add a new error that will be returned in that case to differentiate between the two cases.
2021-01-04 14:05:23 -05:00
hashicorp-ci 1e58b31098 auto-updated agent/uiserver/bindata_assetfs.go from commit 8c9d5ecc2 2021-01-04 18:36:22 +00:00
R.B. Boyer 42dea6f01e
server: deletions of intentions by name using the intention API is now idempotent (#9278)
Restoring a behavior inadvertently changed while fixing #9254
2021-01-04 11:27:00 -06:00
hashicorp-ci 091963c7ea auto-updated agent/uiserver/bindata_assetfs.go from commit 8c0473a62 2021-01-04 16:52:35 +00:00
Michael Montgomery a1748aa2cb Merge branch 'master' into 6074-allow-config-MaxHeaderBytes 2020-12-30 14:14:05 -06:00
Michael Montgomery 519f537b8b Fixed failing tests
Removed use of `NewTestAgent`, per review comment
Removed CLI flag, per review comment
Updated website documentation
Added changelog entry
2020-12-30 14:09:50 -06:00
Daniel Nephin 088831c91e Maybe fix another data race in a test 2020-12-22 18:53:54 -05:00
Daniel Nephin d0f2eca8de Fix one race caused by t.Parallel 2020-12-22 18:27:18 -05:00
hashicorp-ci 5786746ed5 auto-updated agent/uiserver/bindata_assetfs.go from commit 99f102705 2020-12-18 10:43:59 +00:00
hashicorp-ci 098ca1f567 auto-updated agent/uiserver/bindata_assetfs.go from commit 25d6a1277 2020-12-18 09:07:19 +00:00
hashicorp-ci 87b547cfb3 auto-updated agent/uiserver/bindata_assetfs.go from commit c7d917777 2020-12-17 19:01:42 +00:00
hashicorp-ci d302cf769c auto-updated agent/uiserver/bindata_assetfs.go from commit 2e3a66efb 2020-12-17 16:39:34 +00:00
hashicorp-ci 325bca338b auto-updated agent/uiserver/bindata_assetfs.go from commit 635cf4dc9 2020-12-17 16:08:30 +00:00
hashicorp-ci 33fe41ac33 auto-updated agent/uiserver/bindata_assetfs.go from commit 66cc91c69 2020-12-17 15:30:54 +00:00
hashicorp-ci 9ad738b0bf auto-updated agent/uiserver/bindata_assetfs.go from commit 0ca54c608 2020-12-16 16:46:08 +00:00
hashicorp-ci d600dc85ea auto-updated agent/uiserver/bindata_assetfs.go from commit 4404b4f44 2020-12-16 09:23:22 +00:00
hashicorp-ci fff6a07022 auto-updated agent/uiserver/bindata_assetfs.go from commit a919b60f5 2020-12-15 19:36:10 +00:00
hashicorp-ci b7c34f4b71 auto-updated agent/uiserver/bindata_assetfs.go from commit e921b3cf9 2020-12-15 18:33:36 +00:00
hashicorp-ci d0dffb1697 auto-updated agent/uiserver/bindata_assetfs.go from commit 5e150d7f0 2020-12-15 18:19:30 +00:00
John Cowen d3d4c1452a
api: Ensure the internal/ui/service endpoint responds with an array (#9397)
In some circumstances this endpoint will have no results in it (dues to
ACLs, Namespaces or filtering).

This ensures that the response is at least an empty array (`[]`) rather
than `null`
2020-12-15 16:52:00 +00:00
hashicorp-ci 50bad6e642 auto-updated agent/uiserver/bindata_assetfs.go from commit 9f0787197 2020-12-15 16:37:21 +00:00
hashicorp-ci cc3ebef5df auto-updated agent/uiserver/bindata_assetfs.go from commit f111d6b3e 2020-12-15 15:40:26 +00:00
hashicorp-ci de315911b4 auto-updated agent/uiserver/bindata_assetfs.go from commit 14d043e5f 2020-12-14 15:33:47 +00:00
hashicorp-ci 88b34b3996 auto-updated agent/uiserver/bindata_assetfs.go from commit 4e419b9b3 2020-12-14 14:31:06 +00:00
Daniel Nephin b58401480b http: Check HTTPUseCache in a single place
HTTPUseCache is only used is a gate for allowing QueryOptions.UseCache to be enabled. By
moving it to the place where the query options are set, this behaviour is more obvious.

Also remove parseInternal which was an alias for parse.
2020-12-11 14:03:47 -05:00
Daniel Nephin c66a63275f
Merge pull request #9340 from hashicorp/dnephin/skip-slow-tests-with-short
testing: skip slow tests with -short
2020-12-11 13:33:44 -05:00
hashicorp-ci 91c244a830 auto-updated agent/uiserver/bindata_assetfs.go from commit 514270a41 2020-12-11 11:48:26 +00:00
hashicorp-ci 71cd7c6a16 auto-updated agent/uiserver/bindata_assetfs.go from commit 0f7c909f1 2020-12-11 09:44:56 +00:00
hashicorp-ci f827deb8a7 auto-updated agent/uiserver/bindata_assetfs.go from commit 21e5a8f0f 2020-12-11 09:38:09 +00:00
R.B. Boyer f9dcaf7f6b
acl: global tokens created by auth methods now correctly replicate to secondary datacenters (#9351)
Previously the tokens would fail to insert into the secondary's state
store because the AuthMethod field of the ACLToken did not point to a
known auth method from the primary.
2020-12-09 15:22:29 -06:00
hashicorp-ci 6e08ef263e auto-updated agent/uiserver/bindata_assetfs.go from commit 9d8131907 2020-12-09 19:17:30 +00:00
hashicorp-ci 52ca0f2a2b auto-updated agent/uiserver/bindata_assetfs.go from commit a78566e2d 2020-12-09 18:54:02 +00:00
hashicorp-ci bfaf17fa05 auto-updated agent/uiserver/bindata_assetfs.go from commit d6cb2b0d7 2020-12-09 18:45:39 +00:00
Kenia 8bda36c9f4
Create consul version metric with version label (#9350)
* create consul version metric with version label

* agent/agent.go: add pre-release Version as well as label

Co-Authored-By: Radha13 <kumari.radha3@gmail.com>

* verion and pre-release version labels.

* hyphen/- breaks prometheus

* Add Prometheus gauge defintion for version metric

* Add new metric to telemetry docs

Co-authored-by: Radha Kumari <kumari.radha3@gmail.com>
Co-authored-by: Aestek <thib.gilles@gmail.com>
Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>
2020-12-09 09:16:53 -05:00
hashicorp-ci 3f2199d323 auto-updated agent/uiserver/bindata_assetfs.go from commit 27c74f714 2020-12-09 13:12:32 +00:00
hashicorp-ci cdfae9ce63 auto-updated agent/uiserver/bindata_assetfs.go from commit 613be01f4 2020-12-09 09:29:17 +00:00
hashicorp-ci 37613345f2 auto-updated agent/uiserver/bindata_assetfs.go from commit db5283ee2 2020-12-08 15:53:31 +00:00
hashicorp-ci 535ef33bac auto-updated agent/uiserver/bindata_assetfs.go from commit 3be03029f 2020-12-08 09:32:32 +00:00
Daniel Nephin ef0999547a testing: skip slow tests with -short
Add a skip condition to all tests slower than 100ms.

This change was made using `gotestsum tool slowest` with data from the
last 3 CI runs of master.
See https://github.com/gotestyourself/gotestsum#finding-and-skipping-slow-tests

With this change:

```
$ time go test -count=1 -short ./agent
ok      github.com/hashicorp/consul/agent       0.743s

real    0m4.791s

$ time go test -count=1 -short ./agent/consul
ok      github.com/hashicorp/consul/agent/consul        4.229s

real    0m8.769s
```
2020-12-07 13:42:55 -05:00
hashicorp-ci d4478782ae auto-updated agent/uiserver/bindata_assetfs.go from commit 4dfa7622d 2020-12-07 09:24:00 +00:00
hashicorp-ci b734cf54ed auto-updated agent/uiserver/bindata_assetfs.go from commit adbd6c0c8 2020-12-03 09:19:50 +00:00
Kyle Havlovitz 57210a59c3 connect: Fix a case where the active root would get unset even when there wasn't a new one 2020-12-02 11:42:23 -08:00
hashicorp-ci 4ee15914b0 auto-updated agent/uiserver/bindata_assetfs.go from commit e23b5b003 2020-12-02 15:53:16 +00:00
hashicorp-ci a814338374 auto-updated agent/uiserver/bindata_assetfs.go from commit 9ac7bc180 2020-12-02 15:46:59 +00:00
hashicorp-ci 297210ad93 auto-updated agent/uiserver/bindata_assetfs.go from commit a5b9ada9a 2020-12-02 09:49:40 +00:00
hashicorp-ci af9687e0b8 auto-updated agent/uiserver/bindata_assetfs.go from commit cf38309f6 2020-12-01 15:49:06 +00:00
Kyle Havlovitz 91d5d6c586
Merge pull request #9009 from hashicorp/update-secondary-ca
connect: Fix an issue with updating CA config in a secondary datacenter
2020-11-30 14:49:28 -08:00
Kyle Havlovitz c5167cf9c4 Use a buffered channel for CA intermediate renew func 2020-11-30 14:37:24 -08:00
hashicorp-ci 149e1e5f13 auto-updated agent/uiserver/bindata_assetfs.go from commit afe0f2614 2020-11-30 18:47:37 +00:00
hashicorp-ci 45f7de452f auto-updated agent/uiserver/bindata_assetfs.go from commit b5abbf122 2020-11-30 17:33:21 +00:00
hashicorp-ci 58797598dc auto-updated agent/uiserver/bindata_assetfs.go from commit d1ebe8c14 2020-11-30 17:27:35 +00:00
hashicorp-ci c03baa7b57 auto-updated agent/uiserver/bindata_assetfs.go from commit f46ef3e3f 2020-11-30 17:07:25 +00:00
hashicorp-ci 4801228104 auto-updated agent/uiserver/bindata_assetfs.go from commit a59a2f860 2020-11-30 16:57:34 +00:00
hashicorp-ci 2fd62ba8de auto-updated agent/uiserver/bindata_assetfs.go from commit 9cf30e74e 2020-11-30 15:09:43 +00:00
Daniel Nephin 17a86be022
Merge pull request #9284 from hashicorp/dnephin/agent-service-register
local: mark service as InSync when added to local agent state
2020-11-27 15:49:55 -05:00
Daniel Nephin 4c5fab6e00 local: mark service and checks as InSync when added
If the existing service and checks are the same as the new registration.
2020-11-27 15:31:12 -05:00