open-consul/agent/config
Dhia Ayachi cdcb249449
add a rate limiter to config auto-reload (#12490)
* add config watcher to the config package

* add logging to watcher

* add test and refactor to add WatcherEvent.

* add all API calls and fix a bug with recreated files

* add tests for watcher

* remove the unnecessary use of context

* Add debug log and a test for file rename

* use inode to detect if the file is recreated/replaced and only listen to create events.

* tidy ups (#1535)

* tidy ups

* Add tests for inode reconcile

* fix linux vs windows syscall

* fix linux vs windows syscall

* fix windows compile error

* increase timeout

* use ctime ID

* remove remove/creation test as it's a use case that fail in linux

* fix linux/windows to use Ino/CreationTime

* fix the watcher to only overwrite current file id

* fix linter error

* fix remove/create test

* set reconcile loop to 200 Milliseconds

* fix watcher to not trigger event on remove, add more tests

* on a remove event try to add the file back to the watcher and trigger the handler if success

* fix race condition

* fix flaky test

* fix race conditions

* set level to info

* fix when file is removed and get an event for it after

* fix to trigger handler when we get a remove but re-add fail

* fix error message

* add tests for directory watch and fixes

* detect if a file is a symlink and return an error on Add

* rename Watcher to FileWatcher and remove symlink deref

* add fsnotify@v1.5.1

* fix go mod

* do not reset timer on errors, rename OS specific files

* rename New func

* events trigger on write and rename

* add missing test

* fix flaking tests

* fix flaky test

* check reconcile when removed

* delete invalid file

* fix test to create files with different mod time.

* back date file instead of sleeping

* add watching file in agent command.

* fix watcher call to use new API

* add configuration and stop watcher when server stop

* add certs as watched files

* move FileWatcher to the agent start instead of the command code

* stop watcher before replacing it

* save watched files in agent

* add add and remove interfaces to the file watcher

* fix remove to not return an error

* use `Add` and `Remove` to update certs files

* fix tests

* close events channel on the file watcher even when the context is done

* extract `NotAutoReloadableRuntimeConfig` is a separate struct

* fix linter errors

* add Ca configs and outgoing verify to the not auto reloadable config

* add some logs and fix to use background context

* add tests to auto-config reload

* remove stale test

* add tests to changes to config files

* add check to see if old cert files still trigger updates

* rename `NotAutoReloadableRuntimeConfig` to `StaticRuntimeConfig`

* fix to re add both key and cert file. Add test to cover this case.

* review suggestion

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

* add check to static runtime config changes

* fix test

* add changelog file

* fix review comments

* Apply suggestions from code review

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

* update flag description

Co-authored-by: FFMMM <FFMMM@users.noreply.github.com>

* fix compilation error

* add static runtime config support

* fix test

* fix review comments

* fix log test

* Update .changelog/12329.txt

Co-authored-by: Dan Upton <daniel@floppy.co>

* transfer tests to runtime_test.go

* fix filewatcher Replace to not deadlock.

* avoid having lingering locks

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

* split ReloadConfig func

* fix warning message

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

* convert `FileWatcher` into an interface

* fix compilation errors

* fix tests

* extract func for adding and removing files

* add a coalesceTimer with a very small timer

* extract coaelsce Timer and add a shim for testing

* add tests to coalesceTimer fix to send remaining events

* set `coalesceTimer` to 1 Second

* support symlink, fix a nil deref.

* fix compile error

* fix compile error

* refactor file watcher rate limiting to be a Watcher implementation

* fix linter issue

* fix runtime config

* fix runtime test

* fix flaky tests

* fix compile error

* Apply suggestions from code review

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

* fix agent New to return an error if File watcher New return an error

* quit timer loop if ctx is canceled

* Apply suggestions from code review

Co-authored-by: Chris S. Kim <ckim@hashicorp.com>

Co-authored-by: Ashwin Venkatesh <ashwin@hashicorp.com>
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>
Co-authored-by: FFMMM <FFMMM@users.noreply.github.com>
Co-authored-by: Daniel Upton <daniel@floppy.co>
Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
2022-04-04 11:31:39 -04:00
..
testdata add a rate limiter to config auto-reload (#12490) 2022-04-04 11:31:39 -04:00
agent_limits.go config: validate system limits against limits.http_max_conns_per_client (#7434) 2020-04-02 09:22:17 +02:00
builder.go add a rate limiter to config auto-reload (#12490) 2022-04-04 11:31:39 -04:00
builder_oss.go re-run gofmt on 1.17 (#11579) 2021-11-16 12:04:01 -06:00
builder_oss_test.go Rename `master` and `agent_master` ACL tokens in the config file format (#11665) 2021-12-01 21:08:14 +00:00
builder_test.go polish rpc.service.call metric behavior (#12624) 2022-03-31 10:49:37 -07:00
config.go auto-reload configuration when config files change (#12329) 2022-03-31 15:11:49 -04:00
config_oss.go re-run gofmt on 1.17 (#11579) 2021-11-16 12:04:01 -06:00
default.go agent: convert listener config to TLS types (#12522) 2022-03-24 15:32:25 -04:00
default_oss.go re-run gofmt on 1.17 (#11579) 2021-11-16 12:04:01 -06:00
deprecated.go agent: convert listener config to TLS types (#12522) 2022-03-24 15:32:25 -04:00
deprecated_test.go agent: convert listener config to TLS types (#12522) 2022-03-24 15:32:25 -04:00
doc.go Metrics service prefix (#3498) 2017-09-26 17:49:55 -07:00
file_watcher.go add a rate limiter to config auto-reload (#12490) 2022-04-04 11:31:39 -04:00
file_watcher_test.go add a rate limiter to config auto-reload (#12490) 2022-04-04 11:31:39 -04:00
flags.go auto-reload configuration when config files change (#12329) 2022-03-31 15:11:49 -04:00
flags_test.go Revert "Merge pull request #10588 from hashicorp/dnephin/config-fix-ports-grpc" 2021-09-29 12:28:41 -04:00
flagset.go New config parser, HCL support, multiple bind addrs (#3480) 2017-09-25 11:40:42 -07:00
golden_test.go config: Use golden for TestRuntimeConfig_Sanitize 2021-01-11 14:34:03 -05:00
limits.go re-run gofmt on 1.17 (#11579) 2021-11-16 12:04:01 -06:00
limits_windows.go re-run gofmt on 1.17 (#11579) 2021-11-16 12:04:01 -06:00
merge.go Agent Auto Configuration: Configuration Syntax Updates (#8003) 2020-06-16 15:03:22 -04:00
merge_test.go config: Introduce DeprecatedConfig 2021-09-22 13:22:16 -04:00
ratelimited_file_watcher.go add a rate limiter to config auto-reload (#12490) 2022-04-04 11:31:39 -04:00
ratelimited_file_watcher_test.go add a rate limiter to config auto-reload (#12490) 2022-04-04 11:31:39 -04:00
runtime.go add a rate limiter to config auto-reload (#12490) 2022-04-04 11:31:39 -04:00
runtime_oss.go re-run gofmt on 1.17 (#11579) 2021-11-16 12:04:01 -06:00
runtime_oss_test.go re-run gofmt on 1.17 (#11579) 2021-11-16 12:04:01 -06:00
runtime_test.go add a rate limiter to config auto-reload (#12490) 2022-04-04 11:31:39 -04:00
segment_oss.go re-run gofmt on 1.17 (#11579) 2021-11-16 12:04:01 -06:00
segment_oss_test.go re-run gofmt on 1.17 (#11579) 2021-11-16 12:04:01 -06:00