luxolus
/
open-consul
2
0
Fork 0
Code Issues 1 Pull Requests Packages Releases Wiki Activity
open-consul/docs/config
History
John Murret 700c693b33
adding config for request_limits (#15531) 
* server: add placeholder glue for rate limit handler

This commit adds a no-op implementation of the rate-limit handler and
adds it to the `consul.Server` struct and setup code.

This allows us to start working on the net/rpc and gRPC interceptors and
config logic.

* Add handler errors

* Set the global read and write limits

* fixing multilimiter moving packages

* Fix typo

* Simplify globalLimit usage

* add multilimiter and tests

* exporting LimitedEntity

* Apply suggestions from code review

Co-authored-by: John Murret <john.murret@hashicorp.com>

* add config update and rename config params

* add doc string and split config

* Apply suggestions from code review

Co-authored-by: Dan Upton <daniel@floppy.co>

* use timer to avoid go routine leak and change the interface

* add comments to tests

* fix failing test

* add prefix with config edge, refactor tests

* Apply suggestions from code review

Co-authored-by: Dan Upton <daniel@floppy.co>

* refactor to apply configs for limiters under a prefix

* add fuzz tests and fix bugs found. Refactor reconcile loop to have a simpler logic

* make KeyType an exported type

* split the config and limiter trees to fix race conditions in config update

* rename variables

* fix race in test and remove dead code

* fix reconcile loop to not create a timer on each loop

* add extra benchmark tests and fix tests

* fix benchmark test to pass value to func

* server: add placeholder glue for rate limit handler

This commit adds a no-op implementation of the rate-limit handler and
adds it to the `consul.Server` struct and setup code.

This allows us to start working on the net/rpc and gRPC interceptors and
config logic.

* Set the global read and write limits

* fixing multilimiter moving packages

* add server configuration for global rate limiting.

* remove agent test

* remove added stuff from handler

* remove added stuff from multilimiter

* removing unnecessary TODOs

* Removing TODO comment from handler

* adding in defaulting to infinite

* add disabled status in there

* adding in documentation for disabled mode.

* make disabled the default.

* Add mock and agent test

* addig documentation and missing mock file.

* Fixing test TestLoad_IntegrationWithFlags

* updating docs based on PR feedback.

* Updating Request Limits mode to use int based on PR feedback.

* Adding RequestLimits struct so we have a nested struct in ReloadableConfig.

* fixing linting references

* Update agent/consul/rate/handler.go

Co-authored-by: Dan Upton <daniel@floppy.co>

* Update agent/consul/config.go

Co-authored-by: Dan Upton <daniel@floppy.co>

* removing the ignore of the request limits in JSON.  addingbuilder logic to convert any read rate or write rate less than 0 to rate.Inf

* added conversion function to convert request limits object to handler config.

* Updating docs to reflect gRPC and RPC are rate limit and as a result, HTTP requests are as well.

* Updating values for TestLoad_FullConfig() so that they were different and discernable.

* Updating TestRuntimeConfig_Sanitize

* Fixing TestLoad_IntegrationWithFlags test

* putting nil check in place

* fixing rebase

* removing change for missing error checks.  will put in another PR

* Rebasing after default multilimiter config change

* resolving rebase issues

* updating reference for incomingRPCLimiter to use interface

* updating interface

* Updating interfaces

* Fixing mock reference

Co-authored-by: Daniel Upton <daniel@floppy.co>
Co-authored-by: Dhia Ayachi <dhia@hashicorp.com>
 		2022-12-13 13:09:55 -07:00
..
README.md docs: simplify agent docs slugs 2022-04-11 17:38:47 -07:00
checklist-adding-config-fields.md adding config for request_limits (#15531) 2022-12-13 13:09:55 -07:00

README.md

Agent Configuration

The Agent Configuration is the primary mechanism for configuring Consul. Agent Configuration also allows for specifying Config Entries, Services, and Checks that will be loaded when the agent starts.

Most configuration comes from hcl or json files, but some configuration can also be specified using command line flags, and some can be loaded with Auto-Config.

See also the checklist for adding a new field to the configuration.

Code

The Agent Configuration is implemented in agent/config, and the primary entrypoint is Load. Config loading is performed in phases:

  1. Command line flags are used to create a config.LoadOpts and passed to Load.
  2. Load reads all the config files and builds an ordered list of config.Source.
  3. Each config.Source is read to produce a config.Config.
  4. Each config.Config is merged ontop the previous.
  5. A config.RuntimeConfig is produced from the merged config.Config
  6. The config.RuntimeConfig is validated.
  7. Finally a result is returned with the RuntimeConfig and any warnings, or an error.

If Auto-Config is enabled, when it receives the config from the server, the entire process is repeated a second time with the addition config provided as another config.Source.

Default values can be specified in one of the default sources or set when converting from Config to RuntimeConfig in builder.build. Hopefully in the future we should remove one of those ways of setting default values.

Auto-Config

Auto-Config is enabled by the auto_config field in an Agent Configuration file. It is implemented in a couple packages.