Commit Graph

2891 Commits

Author SHA1 Message Date
Daniel Nephin 76ba6c2198 agent: Replace two fields on AddServiceRequest with a func field
The two previous fields were mutually exclusive. They can be represented
with a single function which provides the value.
2021-01-25 18:01:03 -05:00
Daniel Nephin 5e31bdf51a agent: remove serviceRegiration type
Replace with the existing AddServiceRequest struct. These structs are
almost identical. Additionally, the only reason the serviceRegistration
struct existed was to recreate an AddServiceRequest.

By storing and re-using the AddServiceRequest we remove the need to
translate into one type and back to the original type.

We also remove the extra parameters to a function, because those values
are already available from the AddServiceRequest field.

Also a minor optimization to only call tokens.AgentToken() when
necessary. Previous it was being called every time, but the value was
being ignored if the AddServiceRequest had a token.
2021-01-25 18:01:03 -05:00
Daniel Nephin 75b2c55291 agent: remove an a branch in the AddService flow
Handle the decision to use ServiceManager in a single place. Instead of
calling ServiceManager.AddService, then calling back into
addServiceInternal, only call ServiceManager.AddService if we are going
to use it.

This change removes some small duplication and removes a branch from the
AddService flow.
2021-01-25 18:01:03 -05:00
Daniel Nephin 1ce7cdd892 agent: use fields directly, not temp variables
The temprorary variables make it much harder to trace where and how struct
fields are used. If a field is only used a small number of times than
refer to the field directly.
2021-01-25 17:25:04 -05:00
Daniel Nephin 493e987a88 agent: addServiceIternalRequest
Move fields that are only relevant for addServiceInternal onto a new
struct and embed AddServiceRequest.
2021-01-25 17:25:04 -05:00
Daniel Nephin a0b11b4c20 agent: move deprecated AddServiceFromSource to a test file
The method is only used in tests, and only exists for legacy calls.

There was one other package which used this method in tests. Export
the AddServiceRequest and a couple of its fields so the new function can
be used in those tests.
2021-01-25 17:25:03 -05:00
Daniel Nephin de1a80b368 agent: use a single method for Agent.AddService 2021-01-25 17:25:03 -05:00
Daniel Nephin 1c8eca2bfd agent: rename AddService->AddServiceFromSource
In preparation for extracting a single AddService func that accepts a request struct.
2021-01-25 17:25:01 -05:00
Daniel Nephin d7d081f402
Merge pull request #9420 from hashicorp/dnephin/reduce-duplicate-in-catalog-schema
state: reduce interface for Enterprise schema
2021-01-25 17:04:25 -05:00
Chris Boulton 448212060a
connect: add local_request_timeout_ms to configure local_app http timeouts (#9554) 2021-01-25 13:50:00 -06:00
R.B. Boyer 6622185d64
server: use the presense of stored federation state data as a sign that we already activated the federation state feature flag (#9519)
This way we only have to wait for the serf barrier to pass once before
we can make use of federation state APIs Without this patch every
restart needs to re-compute the change.
2021-01-25 13:24:32 -06:00
hashicorp-ci 7c8cfb7a2a auto-updated agent/uiserver/bindata_assetfs.go from commit 09d917618 2021-01-25 19:09:57 +00:00
hashicorp-ci b9d7f04760 auto-updated agent/uiserver/bindata_assetfs.go from commit 88d5e00f9 2021-01-25 18:52:54 +00:00
hashicorp-ci eefa22b79b auto-updated agent/uiserver/bindata_assetfs.go from commit bb9573832 2021-01-25 18:19:10 +00:00
R.B. Boyer e3f5a77ade
server: add OSS stubs supporting validation of source namespaces in service-intentions config entries (#9527) 2021-01-25 11:27:38 -06:00
R.B. Boyer 0247f409a0
server: when wan federating via mesh gateways only do heuristic primary DC bypass on the leader (#9366)
Fixes #9341
2021-01-22 10:03:24 -06:00
hashicorp-ci cae9b96b00 auto-updated agent/uiserver/bindata_assetfs.go from commit 02772e46a 2021-01-20 18:46:55 +00:00
John Cowen d3ecb6d7a0
Fix -ui-content-path without regex (#9569)
* Add templating to inject JSON into an application/json script tag

Plus an external script in order to pick it out and inject the values we
need injecting into ember's environment meta tag.

The UI still uses env style naming (CONSUL_*) but we uses the new style
JSON/golang props behind the scenes.

Co-authored-by: Paul Banks <banks@banksco.de>
2021-01-20 18:40:46 +00:00
hashicorp-ci 30d26fa90e auto-updated agent/uiserver/bindata_assetfs.go from commit ffb6680e7 2021-01-20 17:08:19 +00:00
John Cowen 82a5e6d6fc
api: Ensure the internal/ui/gateway-service-nodes endpoint responds with an array (#9593)
In some circumstances this endpoint will have no results in it (dues to
ACLs, Namespaces, filtering or missing configuration).

This ensures that the response is at least an empty array (`[]`) rather
than `null`
2021-01-20 16:59:02 +00:00
Matt Keeler d871a6f260
Fix flaky test by marking mock expectations as optional (#9596)
These expectations are optional because in a slow CI environment the deadline to cancell the context might occur before the go routine reaches issuing the RPC. Either way we are successfully ensuring context cancellation is working.
2021-01-20 10:58:27 -05:00
hashicorp-ci 884a2d61dc auto-updated agent/uiserver/bindata_assetfs.go from commit 30014ff8f 2021-01-20 15:43:19 +00:00
Freddy 5519051c84
Update topology mapping Refs on all proxy instance deletions (#9589)
* Insert new upstream/downstream mapping to persist new Refs

* Avoid upserting mapping copy if it's a no-op

* Add test with panic repro

* Avoid deleting up/downstreams from inside memdb iterator

* Avoid deleting gateway mappings from inside memdb iterator

* Add CHANGELOG entry

* Tweak changelog entry

Co-authored-by: Paul Banks <banks@banksco.de>
2021-01-20 15:17:26 +00:00
Daniel Nephin 979749d86e state: do not delete from inside an iteration
Deleting from memdb inside an interation can cause a panic from Iterator.Next. This
case is technically safe (for now) because the iterator is using the root radix tree
not a modified one.

However this could break at any time if someone adds an insert or delete to the coordinates table
before this place in the function.

It also sets a bad example, because generally deletes in an interator are not safe. So this
commit uses the pattern we have in other places to move the deletes out of the iteration.
2021-01-19 17:00:07 -05:00
Matt Keeler 2d7a4073e3
Merge pull request #9570 from hashicorp/bugfix/9498 2021-01-19 16:30:04 -05:00
Matt Keeler 2d2ce1fb0c
Ensure that CA initialization does not block leader election.
After fixing that bug I uncovered a couple more:

Fix an issue where we might try to cross sign a cert when we never had a valid root.
Fix a potential issue where reconfiguring the CA could cause either the Vault or AWS PCA CA providers to delete resources that are still required by the new incarnation of the CA.
2021-01-19 15:27:48 -05:00
hashicorp-ci 4b3361fb03 auto-updated agent/uiserver/bindata_assetfs.go from commit be694366a 2021-01-19 15:47:02 +00:00
hashicorp-ci 88ab4cd159 auto-updated agent/uiserver/bindata_assetfs.go from commit 41a4a9f4f 2021-01-19 15:29:55 +00:00
Daniel Nephin 52a1d78e39 state: add a regression test for state store schema
To allow the index to be refactored without accidental changes.

To update the expected value run: 'go test ./agent/consul/state -update'
2021-01-15 18:49:55 -05:00
Daniel Nephin aa21c1ea04 state: reduce interface for Enterprise schema
Using withEnterpriseSchema() we can apply any enterprise schema changes
with a single shim, removing the need to duplicate all of the table
definitions.

Also move all the catalog schemas to a new file to shrink catalog.go a bit.
2021-01-15 18:49:55 -05:00
Daniel Nephin 45c0a71e7e
Merge pull request #8696 from hashicorp/dnephin/fix-load-limits
agent/consul: make Client/Server config reloading more obvious
2021-01-14 17:40:42 -05:00
Daniel Nephin 27a59f86d0
Merge pull request #9436 from hashicorp/dnephin/fix-service-health-req-cache-key
structs: fix caching of ServiceSpecificRequest when ingress=true
2021-01-14 17:26:25 -05:00
Daniel Nephin e8427a48ab agent/consuk: Rename RPCRate -> RPCRateLimit
so that the field name is consistent across config structs.
2021-01-14 17:26:00 -05:00
Daniel Nephin e5320c2db6 agent/consul: make Client/Server config reloading more obvious
I believe this commit also fixes a bug. Previously RPCMaxConnsPerClient was not being re-read from the RuntimeConfig, so passing it to Server.ReloadConfig was never changing the value.

Also improve the test runtime by not doing a lot of unnecessary work.
2021-01-14 17:21:10 -05:00
Daniel Nephin f2b504873a
Merge pull request #9460 from hashicorp/dnephin/fix-data-races
Fix a couple data races in tests
2021-01-14 17:07:01 -05:00
Daniel Nephin ef9d44fdfe structs: fix caching of ServiceSpecificRequest when ingress=true
The field was not being included in the cache info key. This would result in a DNS request for
web.service.consul returning the same result as web.ingress.consul, when those results should
not be the same.
2021-01-14 17:01:40 -05:00
hashicorp-ci 66ec8e5e1f auto-updated agent/uiserver/bindata_assetfs.go from commit 1e30503ec 2021-01-13 09:47:00 +00:00
kevinkengne 18a12d6517
add completeness test for types with CacheInfo method (#9480)
include all fields when fuzzing in tests
split tests by struct type

Ensure the new value for the field is different

fuzzer.Fuzz could produce the same value again in some cases.

Use a custom fuzz function for QueryOptions. That type is an embedded struct in the request types
but only one of the fields is important to include in the cache key.

Move enterpriseMetaField to an oss file so that we can change it in enterprise.
2021-01-12 19:45:46 -05:00
Chris Piraino baad708929
Fix bug in usage metrics when multiple service instances are changed in a single transaction (#9440)
* Fix bug in usage metrics that caused a negative count to occur

There were a couple of instances were usage metrics would do the wrong
thing and result in incorrect counts, causing the count to attempt to
decrement below zero and return an error. The usage metrics did not
account for various places where a single transaction could
delete/update/add multiple service instances at once.

We also remove the error when attempting to decrement below zero, and
instead just make sure we do not accidentally underflow the unsigned
integer. This is a more graceful failure than returning an error and not
allowing a transaction to commit.

* Add changelog
2021-01-12 15:31:47 -06:00
hashicorp-ci 5b92255cf5 auto-updated agent/uiserver/bindata_assetfs.go from commit b86eea4fb 2021-01-12 14:57:52 +00:00
Daniel Nephin 51464763f4 config: remove Builder receiver from funcs that dont use it
This change allows us to re-use these functions in other places without the Builder, and makes it
more explicit about which functions can warn/error and which can not.
2021-01-11 17:41:58 -05:00
Daniel Nephin 78b02663e9 config: Use golden for TestRuntimeConfig_Sanitize
A golden file makes the expected value easier to work with. This change also
removes a number of shims for enterprise and replaces them with a single one
for the golden filename.
2021-01-11 14:34:03 -05:00
Pierre Souchay 70d2da7582
Display a warning when rpc.enable_streaming = true is set on a client (#9530)
* Display a warning when rpc.enable_streaming = true is set on a client

This option has no effect when running as an agent

* Added warning when server starts with use_streaming_backend but without rpc.enable_streaming

* Added unit test
2021-01-08 15:23:23 -05:00
Chris Piraino 2eac571276
Log replication warnings when no error suppression is defined (#9320)
* Log replication warnings when no error suppression is defined

* Add changelog file
2021-01-08 14:03:06 -06:00
Daniel Nephin 6094b73d89
Merge pull request #9250 from hashicorp/dnephin/config-tags-2
config: remove unnecessary json/hcl struct tags
2021-01-08 14:08:34 -05:00
Daniel Nephin d5bdc2f539
Merge pull request #9249 from hashicorp/dnephin/config-tags
config: use fields to detect enterprise-only settings
2021-01-07 19:49:29 -05:00
Daniel Nephin f3b30a7ed6 config: remove unnecessary json/hcl struct tags
Using find/replace with regex

        json:"[^"]+"
        hcl:"[^"]+"
2021-01-07 19:42:56 -05:00
Daniel Nephin ee00d7668c config: Use config fields to warn about enterprise settings
It is no safe to assumes that the mapstructure keys will contain all the keys because some config can be specified
with command line flags or literals.

This change allows us to remove the json marshal/unmarshal cycle for command line flags, which will allow
us to remove all of the hcl/json struct tags on config fields.
2021-01-07 19:31:39 -05:00
Daniel Nephin 0679a23921
Merge pull request #9251 from hashicorp/dnephin/config-cleanup-flags-and-opts
config: move testing shims and cleanup some defaults
2021-01-07 19:11:22 -05:00
Daniel Nephin 942ffa7e02 config: use LiteralConfig for flag values 2021-01-07 18:57:53 -05:00
Daniel Nephin 2d0f33591e config: remove t.Parallel and some unused fields from tests
Use go-cmp for better diffs in tests
2021-01-07 18:57:52 -05:00
Daniel Nephin 31b1addd9e structs: add tests for String() methods
To show that printing one of these IDs works properly now that the String() method
receiver is no longer a pointer.
2021-01-07 18:47:38 -05:00
Daniel Nephin 45f0afcbf4 structs: Fix printing of IDs
These types are used as values (not pointers) in other structs. Using a pointer receiver causes
problems when the value is printed. fmt will not call the String method if it is passed a value
and the String method has a pointer receiver. By using a value receiver the correct string is printed.

Also remove some unused methods.
2021-01-07 18:47:38 -05:00
Daniel Nephin bcfb444a77 Remove an unnecessary else 2021-01-07 18:13:49 -05:00
Daniel Nephin f6543b1651 xds: remove Server.Initialize
Requiring a call to initialize to set a single field is not really substantially different
from having to set that field to a value.
2021-01-07 18:13:48 -05:00
Daniel Nephin bbf1a116f6 xds: Fix data race
TestEnvoy.Close used e.stream.recvCh == nil to indicate the channel had already
been closed, so that TestEnvoy.Close can be called multiple times. The recvCh
was not protected by a lock, so setting it to nil caused a data race with any
goroutine trying to read from the channel.

Instead set the stream to nil. The stream is guarded by a lock, so it does not race.

This change allows us to test the agent/xds package using -race.
2021-01-07 18:13:48 -05:00
Daniel Nephin de226f26e4 xds: Pass in logger
small cleanup in tests
2021-01-07 18:13:48 -05:00
hashicorp-ci 7c5ca27fdd auto-updated agent/uiserver/bindata_assetfs.go from commit e893ba7ea 2021-01-07 19:09:58 +00:00
Daniel Nephin 27c38bfebb
Merge pull request #9213 from hashicorp/dnephin/resolve-tokens-take-2
acl: Remove some unused things and document delegate method
2021-01-06 18:51:51 -05:00
Pierre Souchay c43888c064 Added testing of GRPC with TLS combinations
This ensures that https://github.com/hashicorp/consul/issues/9474 will
not reproduce.
2021-01-06 22:20:23 +01:00
Pierre Souchay 542852786c [Streaming][bugfix] handle TLS signalisation when TLS is disabled on client side
Tnis is an alternative to https://github.com/hashicorp/consul/pull/9494
2021-01-06 17:24:58 +01:00
R.B. Boyer db62541676
acl: use the presence of a management policy in the state store as a sign that we already migrated to v2 acls (#9505)
This way we only have to wait for the serf barrier to pass once before
we can upgrade to v2 acls. Without this patch every restart needs to
re-compute the change, and potentially if a stray older node joins after
a migration it might regress back to v1 mode which would be problematic.
2021-01-05 17:04:27 -06:00
hashicorp-ci 8fc77877be auto-updated agent/uiserver/bindata_assetfs.go from commit 1304dc882 2021-01-05 17:47:53 +00:00
Daniel Nephin 3140c0a343
Merge pull request #9067 from naemono/6074-allow-config-MaxHeaderBytes
Adds option to configure HTTP Server's MaxHeaderBytes
2021-01-05 12:28:27 -05:00
hashicorp-ci e64b90c604 auto-updated agent/uiserver/bindata_assetfs.go from commit a42e844cc 2021-01-05 17:09:19 +00:00
hashicorp-ci 988609fb61 auto-updated agent/uiserver/bindata_assetfs.go from commit 17438020f 2021-01-05 10:11:12 +00:00
Michael Montgomery 5a70c2c7e8 Remove unneeded test 2021-01-04 19:47:13 -06:00
Matt Keeler 3a79b559f9
Special case the error returned when we have a Raft leader but are not tracking it in the ServerLookup (#9487)
This can happen when one other node in the cluster such as a client is unable to communicate with the leader server and sees it as failed. When that happens its failing status eventually gets propagated to the other servers in the cluster and eventually this can result in RPCs returning “No cluster leader” error.

That error is misleading and unhelpful for determing the root cause of the issue as its not raft stability but rather and client -> server networking issue. Therefore this commit will add a new error that will be returned in that case to differentiate between the two cases.
2021-01-04 14:05:23 -05:00
hashicorp-ci 1e58b31098 auto-updated agent/uiserver/bindata_assetfs.go from commit 8c9d5ecc2 2021-01-04 18:36:22 +00:00
R.B. Boyer 42dea6f01e
server: deletions of intentions by name using the intention API is now idempotent (#9278)
Restoring a behavior inadvertently changed while fixing #9254
2021-01-04 11:27:00 -06:00
hashicorp-ci 091963c7ea auto-updated agent/uiserver/bindata_assetfs.go from commit 8c0473a62 2021-01-04 16:52:35 +00:00
Michael Montgomery a1748aa2cb Merge branch 'master' into 6074-allow-config-MaxHeaderBytes 2020-12-30 14:14:05 -06:00
Michael Montgomery 519f537b8b Fixed failing tests
Removed use of `NewTestAgent`, per review comment
Removed CLI flag, per review comment
Updated website documentation
Added changelog entry
2020-12-30 14:09:50 -06:00
Daniel Nephin 088831c91e Maybe fix another data race in a test 2020-12-22 18:53:54 -05:00
Daniel Nephin d0f2eca8de Fix one race caused by t.Parallel 2020-12-22 18:27:18 -05:00
hashicorp-ci 5786746ed5 auto-updated agent/uiserver/bindata_assetfs.go from commit 99f102705 2020-12-18 10:43:59 +00:00
hashicorp-ci 098ca1f567 auto-updated agent/uiserver/bindata_assetfs.go from commit 25d6a1277 2020-12-18 09:07:19 +00:00
hashicorp-ci 87b547cfb3 auto-updated agent/uiserver/bindata_assetfs.go from commit c7d917777 2020-12-17 19:01:42 +00:00
hashicorp-ci d302cf769c auto-updated agent/uiserver/bindata_assetfs.go from commit 2e3a66efb 2020-12-17 16:39:34 +00:00
hashicorp-ci 325bca338b auto-updated agent/uiserver/bindata_assetfs.go from commit 635cf4dc9 2020-12-17 16:08:30 +00:00
hashicorp-ci 33fe41ac33 auto-updated agent/uiserver/bindata_assetfs.go from commit 66cc91c69 2020-12-17 15:30:54 +00:00
hashicorp-ci 9ad738b0bf auto-updated agent/uiserver/bindata_assetfs.go from commit 0ca54c608 2020-12-16 16:46:08 +00:00
hashicorp-ci d600dc85ea auto-updated agent/uiserver/bindata_assetfs.go from commit 4404b4f44 2020-12-16 09:23:22 +00:00
hashicorp-ci fff6a07022 auto-updated agent/uiserver/bindata_assetfs.go from commit a919b60f5 2020-12-15 19:36:10 +00:00
hashicorp-ci b7c34f4b71 auto-updated agent/uiserver/bindata_assetfs.go from commit e921b3cf9 2020-12-15 18:33:36 +00:00
hashicorp-ci d0dffb1697 auto-updated agent/uiserver/bindata_assetfs.go from commit 5e150d7f0 2020-12-15 18:19:30 +00:00
John Cowen d3d4c1452a
api: Ensure the internal/ui/service endpoint responds with an array (#9397)
In some circumstances this endpoint will have no results in it (dues to
ACLs, Namespaces or filtering).

This ensures that the response is at least an empty array (`[]`) rather
than `null`
2020-12-15 16:52:00 +00:00
hashicorp-ci 50bad6e642 auto-updated agent/uiserver/bindata_assetfs.go from commit 9f0787197 2020-12-15 16:37:21 +00:00
hashicorp-ci cc3ebef5df auto-updated agent/uiserver/bindata_assetfs.go from commit f111d6b3e 2020-12-15 15:40:26 +00:00
hashicorp-ci de315911b4 auto-updated agent/uiserver/bindata_assetfs.go from commit 14d043e5f 2020-12-14 15:33:47 +00:00
hashicorp-ci 88b34b3996 auto-updated agent/uiserver/bindata_assetfs.go from commit 4e419b9b3 2020-12-14 14:31:06 +00:00
Daniel Nephin b58401480b http: Check HTTPUseCache in a single place
HTTPUseCache is only used is a gate for allowing QueryOptions.UseCache to be enabled. By
moving it to the place where the query options are set, this behaviour is more obvious.

Also remove parseInternal which was an alias for parse.
2020-12-11 14:03:47 -05:00
Daniel Nephin c66a63275f
Merge pull request #9340 from hashicorp/dnephin/skip-slow-tests-with-short
testing: skip slow tests with -short
2020-12-11 13:33:44 -05:00
hashicorp-ci 91c244a830 auto-updated agent/uiserver/bindata_assetfs.go from commit 514270a41 2020-12-11 11:48:26 +00:00
hashicorp-ci 71cd7c6a16 auto-updated agent/uiserver/bindata_assetfs.go from commit 0f7c909f1 2020-12-11 09:44:56 +00:00
hashicorp-ci f827deb8a7 auto-updated agent/uiserver/bindata_assetfs.go from commit 21e5a8f0f 2020-12-11 09:38:09 +00:00
R.B. Boyer f9dcaf7f6b
acl: global tokens created by auth methods now correctly replicate to secondary datacenters (#9351)
Previously the tokens would fail to insert into the secondary's state
store because the AuthMethod field of the ACLToken did not point to a
known auth method from the primary.
2020-12-09 15:22:29 -06:00
hashicorp-ci 6e08ef263e auto-updated agent/uiserver/bindata_assetfs.go from commit 9d8131907 2020-12-09 19:17:30 +00:00
hashicorp-ci 52ca0f2a2b auto-updated agent/uiserver/bindata_assetfs.go from commit a78566e2d 2020-12-09 18:54:02 +00:00
hashicorp-ci bfaf17fa05 auto-updated agent/uiserver/bindata_assetfs.go from commit d6cb2b0d7 2020-12-09 18:45:39 +00:00