Commit Graph

331 Commits

Author SHA1 Message Date
Pablo Ruiz García 4188769c32
Added new auto_encrypt.grpc_server_tls config option to control AutoTLS enabling of GRPC Server's TLS usage
Fix for #14253

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
2022-08-24 12:31:38 -04:00
Luke Kysow e9960dfdf3
peering: default to false (#13963)
* defaulting to false because peering will be released as beta
* Ignore peering disabled error in bundles cachetype

Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com>
Co-authored-by: freddygv <freddy@hashicorp.com>
Co-authored-by: Matt Keeler <mjkeeler7@gmail.com>
2022-08-01 15:22:36 -04:00
Luke Kysow d21f793b74
peering: add config to enable/disable peering (#13867)
* peering: add config to enable/disable peering

Add config:

```
peering {
  enabled = true
}
```

Defaults to true. When disabled:
1. All peering RPC endpoints will return an error
2. Leader won't start its peering establishment goroutines
3. Leader won't start its peering deletion goroutines
2022-07-22 15:20:21 -07:00
alex 04ec093130
no 1.9 style metrics (#13532)
Signed-off-by: acpana <8968914+acpana@users.noreply.github.com>
2022-06-29 09:46:37 -07:00
Mark Anderson 41446f449a Fixup some more tests
Signed-off-by: Mark Anderson <manderson@hashicorp.com>
2022-06-09 17:04:05 -07:00
cskh df27fa0c84
Retry on bad dogstatsd connection (#13091)
- Introduce a new telemetry configurable parameter retry_failed_connection. User can set the value to true to let consul agent continue its start process on failed connection to datadog server. When set to false, agent will stop on failed start. The default behavior is true.

Co-authored-by: Dan Upton <daniel@floppy.co>
Co-authored-by: Evan Culver <eculver@users.noreply.github.com>
2022-05-19 16:03:46 -04:00
Mark Anderson be059184dd Fix up enterprise version tag.
Changes to how the version string was handled created small regression with the release of consul 1.12.0 enterprise.

Many tools use the Config:Version field reported by the agent/self resource to determine whether Consul is an enterprise or OSS instance, expect something like 1.12.0+ent for enterprise and simply 1.12.0 for OSS. This was accidentally broken during the runup to 1.12.x

This work fixes the value returned by both the self endpoint in ["Config"]["Version"] and the metrics consul.version field.

Signed-off-by: Mark Anderson <manderson@hashicorp.com>
2022-05-09 10:07:19 -07:00
Kyle Havlovitz a2c28526a7
Merge pull request #12685 from hashicorp/http-check-redirect-option
Add a field to disable following redirects on http checks
2022-04-07 11:29:27 -07:00
Kyle Havlovitz b3db499c74 Add a field to disable following redirects on http checks 2022-04-05 16:12:18 -07:00
Dhia Ayachi cdcb249449
add a rate limiter to config auto-reload (#12490)
* add config watcher to the config package

* add logging to watcher

* add test and refactor to add WatcherEvent.

* add all API calls and fix a bug with recreated files

* add tests for watcher

* remove the unnecessary use of context

* Add debug log and a test for file rename

* use inode to detect if the file is recreated/replaced and only listen to create events.

* tidy ups (#1535)

* tidy ups

* Add tests for inode reconcile

* fix linux vs windows syscall

* fix linux vs windows syscall

* fix windows compile error

* increase timeout

* use ctime ID

* remove remove/creation test as it's a use case that fail in linux

* fix linux/windows to use Ino/CreationTime

* fix the watcher to only overwrite current file id

* fix linter error

* fix remove/create test

* set reconcile loop to 200 Milliseconds

* fix watcher to not trigger event on remove, add more tests

* on a remove event try to add the file back to the watcher and trigger the handler if success

* fix race condition

* fix flaky test

* fix race conditions

* set level to info

* fix when file is removed and get an event for it after

* fix to trigger handler when we get a remove but re-add fail

* fix error message

* add tests for directory watch and fixes

* detect if a file is a symlink and return an error on Add

* rename Watcher to FileWatcher and remove symlink deref

* add fsnotify@v1.5.1

* fix go mod

* do not reset timer on errors, rename OS specific files

* rename New func

* events trigger on write and rename

* add missing test

* fix flaking tests

* fix flaky test

* check reconcile when removed

* delete invalid file

* fix test to create files with different mod time.

* back date file instead of sleeping

* add watching file in agent command.

* fix watcher call to use new API

* add configuration and stop watcher when server stop

* add certs as watched files

* move FileWatcher to the agent start instead of the command code

* stop watcher before replacing it

* save watched files in agent

* add add and remove interfaces to the file watcher

* fix remove to not return an error

* use `Add` and `Remove` to update certs files

* fix tests

* close events channel on the file watcher even when the context is done

* extract `NotAutoReloadableRuntimeConfig` is a separate struct

* fix linter errors

* add Ca configs and outgoing verify to the not auto reloadable config

* add some logs and fix to use background context

* add tests to auto-config reload

* remove stale test

* add tests to changes to config files

* add check to see if old cert files still trigger updates

* rename `NotAutoReloadableRuntimeConfig` to `StaticRuntimeConfig`

* fix to re add both key and cert file. Add test to cover this case.

* review suggestion

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

* add check to static runtime config changes

* fix test

* add changelog file

* fix review comments

* Apply suggestions from code review

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

* update flag description

Co-authored-by: FFMMM <FFMMM@users.noreply.github.com>

* fix compilation error

* add static runtime config support

* fix test

* fix review comments

* fix log test

* Update .changelog/12329.txt

Co-authored-by: Dan Upton <daniel@floppy.co>

* transfer tests to runtime_test.go

* fix filewatcher Replace to not deadlock.

* avoid having lingering locks

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

* split ReloadConfig func

* fix warning message

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

* convert `FileWatcher` into an interface

* fix compilation errors

* fix tests

* extract func for adding and removing files

* add a coalesceTimer with a very small timer

* extract coaelsce Timer and add a shim for testing

* add tests to coalesceTimer fix to send remaining events

* set `coalesceTimer` to 1 Second

* support symlink, fix a nil deref.

* fix compile error

* fix compile error

* refactor file watcher rate limiting to be a Watcher implementation

* fix linter issue

* fix runtime config

* fix runtime test

* fix flaky tests

* fix compile error

* Apply suggestions from code review

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

* fix agent New to return an error if File watcher New return an error

* quit timer loop if ctx is canceled

* Apply suggestions from code review

Co-authored-by: Chris S. Kim <ckim@hashicorp.com>

Co-authored-by: Ashwin Venkatesh <ashwin@hashicorp.com>
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>
Co-authored-by: FFMMM <FFMMM@users.noreply.github.com>
Co-authored-by: Daniel Upton <daniel@floppy.co>
Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
2022-04-04 11:31:39 -04:00
Dhia Ayachi 8552efa955
auto-reload configuration when config files change (#12329)
* add config watcher to the config package

* add logging to watcher

* add test and refactor to add WatcherEvent.

* add all API calls and fix a bug with recreated files

* add tests for watcher

* remove the unnecessary use of context

* Add debug log and a test for file rename

* use inode to detect if the file is recreated/replaced and only listen to create events.

* tidy ups (#1535)

* tidy ups

* Add tests for inode reconcile

* fix linux vs windows syscall

* fix linux vs windows syscall

* fix windows compile error

* increase timeout

* use ctime ID

* remove remove/creation test as it's a use case that fail in linux

* fix linux/windows to use Ino/CreationTime

* fix the watcher to only overwrite current file id

* fix linter error

* fix remove/create test

* set reconcile loop to 200 Milliseconds

* fix watcher to not trigger event on remove, add more tests

* on a remove event try to add the file back to the watcher and trigger the handler if success

* fix race condition

* fix flaky test

* fix race conditions

* set level to info

* fix when file is removed and get an event for it after

* fix to trigger handler when we get a remove but re-add fail

* fix error message

* add tests for directory watch and fixes

* detect if a file is a symlink and return an error on Add

* rename Watcher to FileWatcher and remove symlink deref

* add fsnotify@v1.5.1

* fix go mod

* do not reset timer on errors, rename OS specific files

* rename New func

* events trigger on write and rename

* add missing test

* fix flaking tests

* fix flaky test

* check reconcile when removed

* delete invalid file

* fix test to create files with different mod time.

* back date file instead of sleeping

* add watching file in agent command.

* fix watcher call to use new API

* add configuration and stop watcher when server stop

* add certs as watched files

* move FileWatcher to the agent start instead of the command code

* stop watcher before replacing it

* save watched files in agent

* add add and remove interfaces to the file watcher

* fix remove to not return an error

* use `Add` and `Remove` to update certs files

* fix tests

* close events channel on the file watcher even when the context is done

* extract `NotAutoReloadableRuntimeConfig` is a separate struct

* fix linter errors

* add Ca configs and outgoing verify to the not auto reloadable config

* add some logs and fix to use background context

* add tests to auto-config reload

* remove stale test

* add tests to changes to config files

* add check to see if old cert files still trigger updates

* rename `NotAutoReloadableRuntimeConfig` to `StaticRuntimeConfig`

* fix to re add both key and cert file. Add test to cover this case.

* review suggestion

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

* add check to static runtime config changes

* fix test

* add changelog file

* fix review comments

* Apply suggestions from code review

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

* update flag description

Co-authored-by: FFMMM <FFMMM@users.noreply.github.com>

* fix compilation error

* add static runtime config support

* fix test

* fix review comments

* fix log test

* Update .changelog/12329.txt

Co-authored-by: Dan Upton <daniel@floppy.co>

* transfer tests to runtime_test.go

* fix filewatcher Replace to not deadlock.

* avoid having lingering locks

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

* split ReloadConfig func

* fix warning message

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

* convert `FileWatcher` into an interface

* fix compilation errors

* fix tests

* extract func for adding and removing files

Co-authored-by: Ashwin Venkatesh <ashwin@hashicorp.com>
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>
Co-authored-by: FFMMM <FFMMM@users.noreply.github.com>
Co-authored-by: Daniel Upton <daniel@floppy.co>
2022-03-31 15:11:49 -04:00
FFMMM 8b184197b3
polish rpc.service.call metric behavior (#12624) 2022-03-31 10:49:37 -07:00
Eric 91a493efe9 Bump go-control-plane
* `go get cloud.google.com/go@v0.59.0`
* `go get github.com/envoyproxy/go-control-plane@v0.9.9`
* `make envoy-library`
* Bumpprotoc to 3.15.8
2022-03-30 13:11:27 -04:00
Mike Morris 8020fb2098
agent: convert listener config to TLS types (#12522)
* tlsutil: initial implementation of types/TLSVersion

tlsutil: add test for parsing deprecated agent TLS version strings

tlsutil: return TLSVersionInvalid with error

tlsutil: start moving tlsutil cipher suite lookups over to types/tls

tlsutil: rename tlsLookup to ParseTLSVersion, add cipherSuiteLookup

agent: attempt to use types in runtime config

agent: implement b.tlsVersion validation in config builder

agent: fix tlsVersion nil check in builder

tlsutil: update to renamed ParseTLSVersion and goTLSVersions

tlsutil: fixup TestConfigurator_CommonTLSConfigTLSMinVersion

tlsutil: disable invalid config parsing tests

tlsutil: update tests

auto_config: lookup old config strings from base.TLSMinVersion

auto_config: update endpoint tests to use TLS types

agent: update runtime_test to use TLS types

agent: update TestRuntimeCinfig_Sanitize.golden

agent: update config runtime tests to expect TLS types

* website: update Consul agent tls_min_version values

* agent: fixup TLS parsing and compilation errors

* test: fixup lint issues in agent/config_runtime_test and tlsutil/config_test

* tlsutil: add CHACHA20_POLY1305 cipher suites to goTLSCipherSuites

* test: revert autoconfig tls min version fixtures to old format

* types: add TLSVersions public function

* agent: add warning for deprecated TLS version strings

* agent: move agent config specific logic from tlsutil.ParseTLSVersion into agent config builder

* tlsutil(BREAKING): change default TLS min version to TLS 1.2

* agent: move ParseCiphers logic from tlsutil into agent config builder

* tlsutil: remove unused CipherString function

* agent: fixup import for types package

* Revert "tlsutil: remove unused CipherString function"

This reverts commit 6ca7f6f58d268e617501b7db9500113c13bae70c.

* agent: fixup config builder and runtime tests

* tlsutil: fixup one remaining ListenerConfig -> ProtocolConfig

* test: move TLS cipher suites parsing test from tlsutil into agent config builder tests

* agent: remove parseCiphers helper from auto_config_endpoint_test

* test: remove unused imports from tlsutil

* agent: remove resolved FIXME comment

* tlsutil: remove TODO and FIXME in cipher suite validation

* agent: prevent setting inherited cipher suite config when TLS 1.3 is specified

* changelog: add entry for converting agent config to TLS types

* agent: remove FIXME in runtime test, this is covered in builder tests with invalid tls9 value now

* tlsutil: remove config tests for values checked at agent config builder boundary

* tlsutil: remove tls version check from loadProtocolConfig

* tlsutil: remove tests and TODOs for logic checked in TestBuilder_tlsVersion and TestBuilder_tlsCipherSuites

* website: update search link for supported Consul agent cipher suites

* website: apply review suggestions for tls_min_version description

* website: attempt to clean up markdown list formatting for tls_min_version

* website: moar linebreaks to fix tls_min_version formatting

* Revert "website: moar linebreaks to fix tls_min_version formatting"

This reverts commit 38585927422f73ebf838a7663e566ac245f2a75c.

* autoconfig: translate old values for TLSMinVersion

* agent: rename var for translated value of deprecated TLS version value

* Update agent/config/deprecated.go

Co-authored-by: Dan Upton <daniel@floppy.co>

* agent: fix lint issue

* agent: fixup deprecated config test assertions for updated warning

Co-authored-by: Dan Upton <daniel@floppy.co>
2022-03-24 15:32:25 -04:00
Dan Upton 57f0f42733
Support per-listener TLS configuration ⚙️ (#12504)
Introduces the capability to configure TLS differently for Consul's
listeners/ports (i.e. HTTPS, gRPC, and the internal multiplexed RPC
port) which is useful in scenarios where you may want the HTTPS or
gRPC interfaces to present a certificate signed by a well-known/public
CA, rather than the certificate used for internal communication which
must have a SAN in the form `server.<dc>.consul`.
2022-03-18 10:46:58 +00:00
Eric 2d27f9dc47 Add the `connect.enable_serverless_plugin` configuration option. 2022-03-11 09:16:00 -05:00
FFMMM 336a234927
[sync ent] increase segment max limit to 4*64, make configurable (#1424) (#11795)
* commit b6eb27563e747a78b7647d2b5da405e46364cc46
Author: FFMMM <FFMMM@users.noreply.github.com>
Date:   Thu Dec 9 13:53:44 2021 -0800

    increase segment max limit to 4*64, make configurable (#1424)

    Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>

* fix: rename ent changelog file

Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>
2021-12-09 15:36:11 -08:00
Dan Upton 8bc11b08dc
Rename `ACLMasterToken` => `ACLInitialManagementToken` (#11746) 2021-12-07 12:39:28 +00:00
Dan Upton 0230ebb4ef
agent/token: rename `agent_master` to `agent_recovery` (internally) (#11744) 2021-12-07 12:12:47 +00:00
freddygv a1c1e36be7 Allow cross-partition references in disco chain
* Add partition fields to targets like service route destinations
* Update validation to prevent cross-DC + cross-partition references
* Handle partitions when reading config entries for disco chain
* Encode partition in compiled targets
2021-12-06 12:32:19 -07:00
Daniel Nephin 8e2c71528f config: add NoFreelistSync option
# Conflicts:
#	agent/config/testdata/TestRuntimeConfig_Sanitize-enterprise.golden
#	agent/consul/server.go
2021-12-02 16:56:15 -05:00
Dan Upton e1829a8706
Rename `master` and `agent_master` ACL tokens in the config file format (#11665) 2021-12-01 21:08:14 +00:00
FFMMM 27227c0fd2
add root_cert_ttl option for consul connect, vault ca providers (#11428)
* add root_cert_ttl option for consul connect, vault ca providers

Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Chris S. Kim <ckim@hashicorp.com>

* add changelog, pr feedback

Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>

* Update .changelog/11428.txt, more docs

Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>

* Update website/content/docs/agent/options.mdx

Co-authored-by: Kyle Havlovitz <kylehav@gmail.com>

Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>
Co-authored-by: Kyle Havlovitz <kylehav@gmail.com>
2021-11-02 11:02:10 -07:00
Daniel Nephin 0ec2a804df
Merge pull request #10690 from tarat44/h2c-support-in-ping-checks
add support for h2c in h2 ping health checks
2021-11-02 13:53:06 -04:00
FFMMM 67a624a49f
use *telemetry.MetricsPrefix as prometheus.PrometheusOpts.Name (#11290)
Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>
2021-10-21 13:33:01 -07:00
tarat44 e3a18e5203 add test cases for h2ping_use_tls default behavior 2021-10-09 17:12:52 -04:00
tarat44 bda1998175 only set default on H2PingUseTLS if H2PING is set 2021-10-06 22:13:01 -04:00
tarat44 1c1405552a fix formatting 2021-10-05 00:15:04 -04:00
tarat44 f8b47cdfcd change config option to H2PingUseTLS 2021-10-05 00:12:21 -04:00
tarat44 ed4ca3db49 add support for h2c in h2 ping health checks 2021-10-04 22:51:08 -04:00
Daniel Nephin 402d3792b6 Revert "Merge pull request #10588 from hashicorp/dnephin/config-fix-ports-grpc"
This reverts commit 74fb650b6b966588f8faeec26935a858af2b8bb5, reversing
changes made to 58bd8173364effb98b9fd9f9b98d31dd887a9bac.
2021-09-29 12:28:41 -04:00
Daniel Nephin cc46fcc53e config: Move ACLEnableKeyListPolicy to DeprecatedConfig 2021-09-23 15:15:00 -04:00
Daniel Nephin 107c24a68a config: move acl_ttl to DeprecatedConfig 2021-09-23 15:14:59 -04:00
Daniel Nephin 5eb2bebdf8 config: move acl_{default,down}_policy to DeprecatedConfig 2021-09-23 15:14:59 -04:00
Daniel Nephin 408eb0e08e config: Deprecate EnableACLReplication
replaced by ACL.TokenReplication
2021-09-23 15:14:59 -04:00
Daniel Nephin d54db5917f config: move ACL master token and replication to DeprecatedConfig 2021-09-23 15:14:59 -04:00
Daniel Nephin 66453d2de9 config: Move two more fields to DeprecatedConfig
And add a test for deprecated config fields.
2021-09-22 13:23:03 -04:00
Daniel Nephin 23f070e0a1 config: Introduce DeprecatedConfig
This struct allows us to move all the deprecated config options off of
the main config struct, and keeps all the deprecation logic in a single
place, instead of spread across 3+ places.
2021-09-22 13:22:16 -04:00
Daniel Nephin 44d91ea56f
Add failures_before_warning to checks (#10969)
Signed-off-by: Jakub Sokołowski <jakub@status.im>

* agent: add failures_before_warning setting

The new setting allows users to specify the number of check failures
that have to happen before a service status us updated to be `warning`.
This allows for more visibility for detected issues without creating
alerts and pinging administrators. Unlike the previous behavior, which
caused the service status to not update until it reached the configured
`failures_before_critical` setting, now Consul updates the Web UI view
with the `warning` state and the output of the service check when
`failures_before_warning` is breached.

The default value of `FailuresBeforeWarning` is the same as the value of
`FailuresBeforeCritical`, which allows for retaining the previous default
behavior of not triggering a warning.

When `FailuresBeforeWarning` is set to a value higher than that of
`FailuresBeforeCritical it has no effect as `FailuresBeforeCritical`
takes precedence.

Resolves: https://github.com/hashicorp/consul/issues/10680

Signed-off-by: Jakub Sokołowski <jakub@status.im>

Co-authored-by: Jakub Sokołowski <jakub@status.im>
2021-09-14 12:47:52 -04:00
R.B. Boyer 6b5a58de50
acl: some acl authz refactors for nodes (#10909) 2021-08-25 13:43:11 -05:00
Dhia Ayachi f766b6dff7
oss portion of ent #1069 (#10883) 2021-08-20 12:57:45 -04:00
Daniel Nephin 09ae0ab94a acl: make ACLDisabledTTL a constant
This field was never user-configurable. We always overwrote the value with 120s from
NonUserSource. However, we also never copied the value from RuntimeConfig to consul.Config,
So the value in NonUserSource was always ignored, and we used the default value of 30s
set by consul.DefaultConfig.

All of this code is an unnecessary distraction because a user can not actually configure
this value.

This commit removes the fields and uses a constant value instad. Someone attempting to set
acl.disabled_ttl in their config will now get an error about an unknown field, but previously
the value was completely ignored, so the new behaviour seems more correct.

We have to keep this field in the AutoConfig response for backwards compatibility, but the value
will be ignored by the client, so it doesn't really matter what value we set.
2021-08-17 13:34:18 -04:00
Daniel Nephin 0d69b49f41 config: remove ACLResolver settings from RuntimeConfig 2021-08-17 13:32:52 -04:00
Daniel Nephin 047abdd73c acl: remove ACLDatacenter
This field has been unnecessary for a while now. It was always set to the same value
as PrimaryDatacenter. So we can remove the duplicate field and use PrimaryDatacenter
directly.

This change was made by GoLand refactor, which did most of the work for me.
2021-08-06 18:27:00 -04:00
Chris S. Kim 6341183a84
agent: update proxy upstreams to inherit namespace from service (#10688) 2021-07-26 17:12:29 -04:00
R.B. Boyer 62ac98b564
agent/structs: add a bunch more EnterpriseMeta helper functions to help with partitioning (#10669) 2021-07-22 13:20:45 -05:00
Blake Covarrubias 441a6c9969
Add DNS recursor strategy option (#10611)
This change adds a new `dns_config.recursor_strategy` option which
controls how Consul queries DNS resolvers listed in the `recursors`
config option. The supported options are `sequential` (default), and
`random`.

Closes #8807

Co-authored-by: Blake Covarrubias <blake@covarrubi.as>
Co-authored-by: Priyanka Sengupta <psengupta@flatiron.com>
2021-07-19 15:22:51 -07:00
Daniel Nephin b4ab87111c
Merge pull request #10567 from hashicorp/dnephin/config-unexport-build
config: unexport the remaining builder methods
2021-07-15 12:05:19 -04:00
Daniel Nephin 2ab6be6a88 config: update GRPCPort and addr in runtime config 2021-07-09 12:31:53 -04:00
Daniel Nephin e8e5defc71 config: unexport the remaining builder methods
And remove BuildAndValidate. This commit completes some earlier work to reduce the config
interface a single Load function.

The last remaining test was converted to use Load instad of BuildAndValidate.
2021-07-06 18:42:09 -04:00