Commit Graph

19383 Commits

Author SHA1 Message Date
John Murret 2a0aeb2349
Rate Limit Handler - ensure rate limiting is not in the code path when not configured (#15819)
* Rate limiting handler - ensure configuration has changed before modifying limiters

* Updating test to validate arguments to UpdateConfig

* Removing duplicate test.  Updating mock.

* Renaming NullRateLimiter to NullRequestLimitsHandler

* Rate Limit Handler - ensure rate limiting is not in the code path when not configured

* Update agent/consul/rate/handler.go

Co-authored-by: Dhia Ayachi <dhia@hashicorp.com>

* formatting handler.go

* Rate limiting handler - ensure configuration has changed before modifying limiters

* Updating test to validate arguments to UpdateConfig

* Removing duplicate test.  Updating mock.

* adding logging for when UpdateConfig is called but the config has not changed.

* Update agent/consul/rate/handler.go

Co-authored-by: Dhia Ayachi <dhia@hashicorp.com>

* Update agent/consul/rate/handler_test.go

Co-authored-by: Dan Upton <daniel@floppy.co>

* modifying existing variable name based on pr feedback

* updating a broken merge conflict;

Co-authored-by: Dhia Ayachi <dhia@hashicorp.com>
Co-authored-by: Dan Upton <daniel@floppy.co>
2022-12-20 15:00:22 -07:00
John Murret 8c33d7cc0e
Rate limiting handler - ensure configuration has changed before modifying limiters (#15805)
* Rate limiting handler - ensure configuration has changed before modifying limiters

* Updating test to validate arguments to UpdateConfig

* Removing duplicate test.  Updating mock.

* adding logging for when UpdateConfig is called but the config has not changed.

* Update agent/consul/rate/handler.go

Co-authored-by: Dhia Ayachi <dhia@hashicorp.com>

Co-authored-by: Dhia Ayachi <dhia@hashicorp.com>
2022-12-20 14:12:03 -07:00
Vladislav Sharapov a0a8a205c5
Update service-discovery.mdx (#15832)
* Update service-discovery.mdx

* Update website/content/docs/concepts/service-discovery.mdx

Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
2022-12-20 13:54:33 -06:00
Michael Wilkerson ebed9e048f
Enhancement: Consul Compatibility Checking (#15818)
* add functions for returning the max and min Envoy major versions
- added an UnsupportedEnvoyVersions list
- removed an unused error from TestDetermineSupportedProxyFeaturesFromString
- modified minSupportedVersion to use the function for getting the Min Envoy major version. Using just the major version without the patch is equivalent to using `.0`

* added a function for executing the envoy --version command
- added a new exec.go file to not be locked to unix system

* added envoy version check when using consul connect envoy

* added changelog entry

* added docs change
2022-12-20 09:58:19 -08:00
Derek Menteer e25f7313e4
Fix incorrect protocol check on discovery chains with peer targets. (#15833) 2022-12-20 10:15:03 -06:00
trujillo-adam 583ac64c27
fixed bad markdown in network segments usage (#15852) 2022-12-19 16:23:57 -08:00
trujillo-adam 1732c60a8f
Docs/network segments tutorial docs conversion (#15829)
* added a NS folder and refactored main page into the overview page

* added NS usage page to NS folder

* updated links to NS docs

* updated nav

* addressed feedback from review
2022-12-19 15:35:05 -08:00
Semir Patel 971089482c
Map net/rpc endpoints to a read/write/exempt op for rate-limiting (#15825)
Also fixed TestRequestRecorder flaky tests due to loss of precision in elapsed time in the test.
2022-12-19 16:04:52 -06:00
Nitya Dhanushkodi 8386bf19bf
extensions: refactor serverless plugin to use extensions from config entry fields (#15817)
docs: update config entry docs and the Lambda manual registration docs

Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com>
Co-authored-by: Eric <eric@haberkorn.co>
2022-12-19 12:19:37 -08:00
Chris S. Kim 87485d05bd
Error out `consul connect envoy` if agent explicitly disabled grpc (#15794)
Co-authored-by: Paul Glass <pglass@hashicorp.com>
2022-12-19 14:37:27 -05:00
David Yu 916d4df29c
Add Service Mesh work stream to JIRA API call (#15845) 2022-12-19 19:10:30 +00:00
Chris S. Kim f129a6a9d0
Break instead (#15844) 2022-12-19 11:53:05 -07:00
Tu Nguyen 032594f5f2
Update docs to reflect vault and consul compatibility (#15826)
* update docs to reflect vault and consul compatibility

* Update website/content/docs/connect/ca/vault.mdx

Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com>

* Apply suggestions from code review

* Apply suggestions from code review

Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com>

Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com>
2022-12-19 10:32:25 -08:00
Chris S. Kim f8868c7ccf
Add custom balancer to always remove subConns (#15701)
The new balancer is a patched version of gRPC's default pick_first balancer
which removes the behavior of preserving the active subconnection if
a list of new addresses contains the currently active address.
2022-12-19 17:39:31 +00:00
cskh eff52f73be
Change field to pointer so it will be parsed as nil (#15831) 2022-12-19 10:25:36 -05:00
Andrew Stucki 1ff0906a3e
Add async reconciliation controller subpackage (#15534)
* Add async reconciliation controller subpackage

* Address initial feedback

* Add tests for panic assertions

* Fix comment
2022-12-16 16:49:26 -05:00
Dhia Ayachi a1ceeff461
add missing code and fix enterprise specific code (#15375)
* add missing code and fix enterprise specific code

* fix retry

* fix flaky tests

* fix linter error in test
2022-12-16 16:31:05 -05:00
David Yu 6ac8193876
JIRA: Small fix to search plugin for JIRA issues (#15816) 2022-12-15 21:32:46 +00:00
cskh b75afa46f5
Upgrade test: test peering upgrade from an old version of consul (#15768)
* upgrade test: test peering upgrade from an old version of consul

NET-1809
2022-12-15 16:31:12 -05:00
Semir Patel d9fb26061c
Force installation of shadow-utils for access to groupadd executable in ubi image (#15812)
Removing curl inadvertently dropped the transitive dependency on shadow-utils
which is required for the groupadd executable. Since curl-minimal does not
have this dependency, make it explicit as part of `microdnf install`.
2022-12-15 12:42:22 -06:00
Dhia Ayachi 108653b9e3
add log-drop package (#15670)
* add log-drop package

* refactor to extract level

* extract metrics

* Apply suggestions from code review

Co-authored-by: Dan Upton <daniel@floppy.co>

* fix compile errors

* change to implement a log sink

* fix tests to remove sleep

* rename and add go docs

* fix expending variadic

Co-authored-by: Dan Upton <daniel@floppy.co>
2022-12-15 12:52:48 -05:00
Semir Patel 8581fc7100
Fix curl vs curl-minimail install conflict in ubi9 image (#15808)
ubi9 already has curl-minimal installed. Attempting to install curl
results in a microdnf conflict and errors out.
2022-12-15 11:00:39 -06:00
Dan Stough 480a7d2ff6
docs: update changelog from 1.14.3, 1.13.5, 1.12.8 (#15804) 2022-12-14 18:47:35 -05:00
David Yu 32d7698a66
add .yaml extension to enable Github Actions workflow (#15799) 2022-12-14 21:46:07 +00:00
Pier-Luc Caron St-Pierre 97b859a690
connect: Add support for ConsulResolver to specifies a filter expression (#15659)
* connect: Add support for ConsulResolver to specifies a filter expression
2022-12-14 12:41:07 -08:00
Paul Glass 62df6a7513
Deprecate -join and -join-wan (#15598) 2022-12-14 20:28:25 +00:00
David Yu a1973f88a5
Setup JIRA sync for Community Issues and PRs via Github actions (#15778)
* Setup JIRA sync via Github actions
2022-12-14 20:07:10 +00:00
Dhia Ayachi 11f245f24f
Server side rate limiter: handle the race condition for limiters tree write in multilimiter (#15767)
* change to perform all tree writes in the same go routine to avoid race condition.

* rename runStoreOnce to reconcile

* Apply suggestions from code review

Co-authored-by: Dan Upton <daniel@floppy.co>

* reduce nesting

Co-authored-by: Dan Upton <daniel@floppy.co>
2022-12-14 17:32:11 +00:00
Semir Patel 1f82e82e04
Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700) 2022-12-14 09:24:22 -06:00
David Yu 456d50dfaa
Dockerfile: bump UBI base image to 9.1 (#15386) 2022-12-14 09:52:19 -05:00
Paul Glass 237b043d91
Update compat table for ECS (#15782) 2022-12-13 16:15:30 -06:00
John Murret 700c693b33
adding config for request_limits (#15531)
* server: add placeholder glue for rate limit handler

This commit adds a no-op implementation of the rate-limit handler and
adds it to the `consul.Server` struct and setup code.

This allows us to start working on the net/rpc and gRPC interceptors and
config logic.

* Add handler errors

* Set the global read and write limits

* fixing multilimiter moving packages

* Fix typo

* Simplify globalLimit usage

* add multilimiter and tests

* exporting LimitedEntity

* Apply suggestions from code review

Co-authored-by: John Murret <john.murret@hashicorp.com>

* add config update and rename config params

* add doc string and split config

* Apply suggestions from code review

Co-authored-by: Dan Upton <daniel@floppy.co>

* use timer to avoid go routine leak and change the interface

* add comments to tests

* fix failing test

* add prefix with config edge, refactor tests

* Apply suggestions from code review

Co-authored-by: Dan Upton <daniel@floppy.co>

* refactor to apply configs for limiters under a prefix

* add fuzz tests and fix bugs found. Refactor reconcile loop to have a simpler logic

* make KeyType an exported type

* split the config and limiter trees to fix race conditions in config update

* rename variables

* fix race in test and remove dead code

* fix reconcile loop to not create a timer on each loop

* add extra benchmark tests and fix tests

* fix benchmark test to pass value to func

* server: add placeholder glue for rate limit handler

This commit adds a no-op implementation of the rate-limit handler and
adds it to the `consul.Server` struct and setup code.

This allows us to start working on the net/rpc and gRPC interceptors and
config logic.

* Set the global read and write limits

* fixing multilimiter moving packages

* add server configuration for global rate limiting.

* remove agent test

* remove added stuff from handler

* remove added stuff from multilimiter

* removing unnecessary TODOs

* Removing TODO comment from handler

* adding in defaulting to infinite

* add disabled status in there

* adding in documentation for disabled mode.

* make disabled the default.

* Add mock and agent test

* addig documentation and missing mock file.

* Fixing test TestLoad_IntegrationWithFlags

* updating docs based on PR feedback.

* Updating Request Limits mode to use int based on PR feedback.

* Adding RequestLimits struct so we have a nested struct in ReloadableConfig.

* fixing linting references

* Update agent/consul/rate/handler.go

Co-authored-by: Dan Upton <daniel@floppy.co>

* Update agent/consul/config.go

Co-authored-by: Dan Upton <daniel@floppy.co>

* removing the ignore of the request limits in JSON.  addingbuilder logic to convert any read rate or write rate less than 0 to rate.Inf

* added conversion function to convert request limits object to handler config.

* Updating docs to reflect gRPC and RPC are rate limit and as a result, HTTP requests are as well.

* Updating values for TestLoad_FullConfig() so that they were different and discernable.

* Updating TestRuntimeConfig_Sanitize

* Fixing TestLoad_IntegrationWithFlags test

* putting nil check in place

* fixing rebase

* removing change for missing error checks.  will put in another PR

* Rebasing after default multilimiter config change

* resolving rebase issues

* updating reference for incomingRPCLimiter to use interface

* updating interface

* Updating interfaces

* Fixing mock reference

Co-authored-by: Daniel Upton <daniel@floppy.co>
Co-authored-by: Dhia Ayachi <dhia@hashicorp.com>
2022-12-13 13:09:55 -07:00
Dan Stough b7c51a31c4
feat: add access logging API to proxy defaults (#15780) 2022-12-13 14:52:18 -05:00
cskh 3e37a449c8
feat(ingress-gateway): support outlier detection of upstream service for ingress gateway (#15614)
* feat(ingress-gateway): support outlier detection of upstream service for ingress gateway

* changelog

Co-authored-by: Eric Haberkorn <erichaberkorn@gmail.com>
2022-12-13 11:51:37 -05:00
Derek Menteer 50a5549f8a
Fix DialedDirectly configuration for Consul dataplane. (#15760)
Fix DialedDirectly configuration for Consul dataplane.
2022-12-13 09:16:31 -06:00
Dan Upton c73707ca3c
grpc: add rate-limiting middleware (#15550)
Implements the gRPC middleware for rate-limiting as a tap.ServerInHandle
function (executed before the request is unmarshaled).

Mappings between gRPC methods and their operation type are generated by
a protoc plugin introduced by #15564.
2022-12-13 15:01:56 +00:00
Dan Upton 4894848993
server: add placeholder glue for rate limit handler (#15539)
Adds a no-op implementation of the rate-limit handler and exposes
it on the consul.Server struct.

It allows us to start working on the net/rpc and gRPC interceptors
and config (re)loading logic, without having to implement the full
handler up-front.

Co-authored-by: John Murret <john.murret@hashicorp.com>
Co-authored-by: Dhia Ayachi <dhia@hashicorp.com>
2022-12-13 11:41:54 +00:00
Kyle Schochenmaier 83fb137d4f
add changelog for enterprise 3846 (#15773) 2022-12-12 15:08:02 -06:00
Freddy 45e2d9655a
Update changelog checker to handle ent-only entry (#15774) 2022-12-12 20:36:42 +00:00
John Murret fe0432ade5
agent: Fix assignment of error when auto-reloading cert and key file changes. (#15769)
* Adding the setting of errors missing in config file watcher code in agent.

* add changelog
2022-12-12 12:24:39 -07:00
Freddy 990f786d2f
Add initial dev docs on config entries (#15717) 2022-12-12 12:02:18 -07:00
John Murret b8a9ea77f0
Fixing CLI instructions so that the installing a cluster that is embedded in the mac instructions is moved outside of the tabbed instructions. (#15736)
* Fixing CLI instructions so that the installing a cluster that is embedded in the mac instructions is moved outside of the tabbed instructions.
Co-authored-by: boruszak <jeffrey.boruszak@hashicorp.com>
Co-authored-by: David Yu <dyu@hashicorp.com>
2022-12-10 18:21:45 +00:00
Geoffrey Grosenbach 6aa98ef755
Remove mention of enterprise license form (#15752)
The license form is no longer available so this recommends contacting a support representative instead.
2022-12-09 14:02:25 -08:00
R.B. Boyer a52a774c09
test: remove variable shadowing in TestDNS_ServiceLookup_ARecordLimits (#15740) 2022-12-09 10:19:02 -06:00
Jared Kirschner ec1562d7cf
docs: how to launch envoy when ACLs enabled (#15497) 2022-12-08 20:15:58 -05:00
Jeff Boruszak 5f91df3642
docs: Enterprise Feature Table update (#15738)
* Table update

* Old table re-added and alphabetized

* Update website/content/docs/enterprise/index.mdx

Co-authored-by: Tu Nguyen <im2nguyen@users.noreply.github.com>

Co-authored-by: Tu Nguyen <im2nguyen@users.noreply.github.com>
2022-12-08 16:49:49 -06:00
Dan Stough c01b9d5bfe
[OSS] security: update x/net module (#15737)
Co-authored-by: Dhia Ayachi <dhia@hashicorp.com>
2022-12-08 16:45:44 -05:00
Eric Haberkorn 5dd131fee8
Remove the `connect.enable_serverless_plugin` agent configuration option (#15710) 2022-12-08 14:46:42 -05:00
Dhia Ayachi b459d58e8d
add multilimiter and tests (#15467)
* add multilimiter and tests

* exporting LimitedEntity

* go mod tidy

* Apply suggestions from code review

Co-authored-by: John Murret <john.murret@hashicorp.com>

* add config update and rename config params

* add doc string and split config

* Apply suggestions from code review

Co-authored-by: Dan Upton <daniel@floppy.co>

* use timer to avoid go routine leak and change the interface

* add comments to tests

* fix failing test

* add prefix with config edge, refactor tests

* Apply suggestions from code review

Co-authored-by: Dan Upton <daniel@floppy.co>

* refactor to apply configs for limiters under a prefix

* add fuzz tests and fix bugs found. Refactor reconcile loop to have a simpler logic

* make KeyType an exported type

* split the config and limiter trees to fix race conditions in config update

* rename variables

* fix race in test and remove dead code

* fix reconcile loop to not create a timer on each loop

* add extra benchmark tests and fix tests

* fix benchmark test to pass value to func

* use a separate go routine to write limiters (#15643)

* use a separate go routine to write limiters

* Add updating limiter when another limiter is created

* fix waiter to be a ticker, so we commit more than once.

* fix tests and add tests for coverage

* unexport members and add tests

* make UpdateConfig thread safe and multi call to Run safe

* replace swith with if

* fix review comments

* replace time.sleep with retries

* fix flaky test and remove unnecessary init

* fix test races

* remove unnecessary negative test case

* remove fixed todo

Co-authored-by: John Murret <john.murret@hashicorp.com>
Co-authored-by: Dan Upton <daniel@floppy.co>
2022-12-08 14:42:07 -05:00
Kevin Wang 8fc7fdbdc7
ignore `mktg-.*` branches for CI tests (#15651) 2022-12-08 12:06:30 -05:00