Commit graph

373 commits

Author SHA1 Message Date
Daniel Nephin e6badb3129 config: make TestLoad_IntegrationWithFlags easier to work with
Replace the large table of tests with individual calls to run(). By using
runCase, failure messages will include the line number for the test case, as
well as a line number from the test functions.

Example:
=== FAIL: agent/config TestLoad_IntegrationWithFlags/failing_case (0.01s)
    runtime_test.go:4721: case: failing case
    runtime_test.go:4864: error "data_dir cannot be empty" does not contain "I expected this error"

Previous:
    runtime_test.go:4864: error "data_dir cannot be empty" does not contain "I expected this error"

Without the line number to the testCase data, debugging these tests is
difficult. It is impossible to jump directly to the test case, and
difficult to find the location because of many similarly named cases.
2021-02-16 14:40:16 -05:00
Daniel Nephin db53954a3f move test case iteration to caller
To make the test case logic more obvious
2021-02-16 14:40:16 -05:00
Daniel Nephin 7499e38f2b config: remove test cases for impossible scenarios
AEInterval is overridden by NonUserSource, so there is no way for a user
to set this value. These two cases represented impossible real world
scenarios.

Instead the test is replaced with one that shows that the AEInterval can
not be set by config.

This change allows us to remove the hcltail and jsontail fields from
testCase
2021-02-16 14:40:15 -05:00
Daniel Nephin 5f2c358255 Fix error checking
There were two identical if lines, remove one and use a switch/case so the compiler
can catch this problem.
2021-02-16 14:40:15 -05:00
Daniel Nephin 1ff2525cee config: move testing shims to LoadOpts
This will allow us to eventually port these tests to use config.Load instead of
unexported functions.
2021-02-16 14:40:15 -05:00
Daniel Nephin a877ec79b6 Remove patchActual, does not appear to be necessary
also small rename and fix comment
2021-02-16 14:40:15 -05:00
Daniel Nephin fc88c28bb9 config: make config.TestLoad_FullConfig use config.Load
This commit makes a number of changes that should make
TestLoad_FullConfig easier to work with, and make the test more like
real world scenarios.

* use separate files in testdata/ dir to store the config source.
  Separate files are much easier to edit because editors can syntax
  highlight json/hcl, and it makes strings easier to find. Previously
  trying to find strings would match strings used in other tests.
* use the exported config.Load interface instead of internal NewBuilder
  and BuildAndValidate.
* remove the tail config overrides, which are only necessary with
  nonZero works.
2021-01-27 17:51:53 -05:00
Daniel Nephin aa42ff5d81 config: Unexport Builder and NewBuilder
This type and constructor are implementation details of config loading.
All callers should use config.Load.
2021-01-27 17:41:53 -05:00
Daniel Nephin 08b2039f5a config: replace calls to config.NewBuilder with config.Load
This is another incremental change to reduce config loading to a single
small interface. All calls to NewBuilder can be replaced with Load.
2021-01-27 17:34:43 -05:00
Daniel Nephin bd545963cc config: improve the interface of Load
This commit reduces the interface to Load() a bit, in preparation for
unexporting NewBuilder and having everything call Load.

The three arguments are reduced to a single argument by moving the other
two into the options struct.

The three return values are reduced to two by moving the RuntimeConfig
and Warnings into a LoadResult struct.
2021-01-27 17:34:43 -05:00
Daniel Nephin 63ba836257
Merge pull request #9252 from hashicorp/dnephin/config-unmethod
config: remove Builder receiver from funcs that dont use it
2021-01-27 17:31:17 -05:00
Daniel Nephin e8427a48ab agent/consuk: Rename RPCRate -> RPCRateLimit
so that the field name is consistent across config structs.
2021-01-14 17:26:00 -05:00
Daniel Nephin 51464763f4 config: remove Builder receiver from funcs that dont use it
This change allows us to re-use these functions in other places without the Builder, and makes it
more explicit about which functions can warn/error and which can not.
2021-01-11 17:41:58 -05:00
Daniel Nephin 78b02663e9 config: Use golden for TestRuntimeConfig_Sanitize
A golden file makes the expected value easier to work with. This change also
removes a number of shims for enterprise and replaces them with a single one
for the golden filename.
2021-01-11 14:34:03 -05:00
Pierre Souchay 70d2da7582
Display a warning when rpc.enable_streaming = true is set on a client (#9530)
* Display a warning when rpc.enable_streaming = true is set on a client

This option has no effect when running as an agent

* Added warning when server starts with use_streaming_backend but without rpc.enable_streaming

* Added unit test
2021-01-08 15:23:23 -05:00
Daniel Nephin 6094b73d89
Merge pull request #9250 from hashicorp/dnephin/config-tags-2
config: remove unnecessary json/hcl struct tags
2021-01-08 14:08:34 -05:00
Daniel Nephin d5bdc2f539
Merge pull request #9249 from hashicorp/dnephin/config-tags
config: use fields to detect enterprise-only settings
2021-01-07 19:49:29 -05:00
Daniel Nephin f3b30a7ed6 config: remove unnecessary json/hcl struct tags
Using find/replace with regex

        json:"[^"]+"
        hcl:"[^"]+"
2021-01-07 19:42:56 -05:00
Daniel Nephin ee00d7668c config: Use config fields to warn about enterprise settings
It is no safe to assumes that the mapstructure keys will contain all the keys because some config can be specified
with command line flags or literals.

This change allows us to remove the json marshal/unmarshal cycle for command line flags, which will allow
us to remove all of the hcl/json struct tags on config fields.
2021-01-07 19:31:39 -05:00
Daniel Nephin 0679a23921
Merge pull request #9251 from hashicorp/dnephin/config-cleanup-flags-and-opts
config: move testing shims and cleanup some defaults
2021-01-07 19:11:22 -05:00
Daniel Nephin 942ffa7e02 config: use LiteralConfig for flag values 2021-01-07 18:57:53 -05:00
Daniel Nephin 2d0f33591e config: remove t.Parallel and some unused fields from tests
Use go-cmp for better diffs in tests
2021-01-07 18:57:52 -05:00
Michael Montgomery 5a70c2c7e8 Remove unneeded test 2021-01-04 19:47:13 -06:00
Michael Montgomery a1748aa2cb Merge branch 'master' into 6074-allow-config-MaxHeaderBytes 2020-12-30 14:14:05 -06:00
Michael Montgomery 519f537b8b Fixed failing tests
Removed use of `NewTestAgent`, per review comment
Removed CLI flag, per review comment
Updated website documentation
Added changelog entry
2020-12-30 14:09:50 -06:00
Daniel Nephin ef0999547a testing: skip slow tests with -short
Add a skip condition to all tests slower than 100ms.

This change was made using `gotestsum tool slowest` with data from the
last 3 CI runs of master.
See https://github.com/gotestyourself/gotestsum#finding-and-skipping-slow-tests

With this change:

```
$ time go test -count=1 -short ./agent
ok      github.com/hashicorp/consul/agent       0.743s

real    0m4.791s

$ time go test -count=1 -short ./agent/consul
ok      github.com/hashicorp/consul/agent/consul        4.229s

real    0m8.769s
```
2020-12-07 13:42:55 -05:00
Hans Hasselberg 8c5c6e77ec
fix serf_wan documentation (#9289)
WAN config is different than LAN config, source of truth is
f72d2042a8/config.go (L315-L326)
and now the docs are correct.
2020-11-27 20:49:43 +01:00
Daniel Nephin 1987a1eca0 config: remove unused const 2020-11-20 19:17:12 -05:00
Daniel Nephin 8e783fb37b config: move testing shims to BuilderOpts
And remove the devMode field from builder.

This change helps make the Builder state more explicit by moving inputs to the BuilderOps struct,
leaving only fields that can change during Builder.Build on the Builder struct.
2020-11-20 18:31:10 -05:00
Daniel Nephin fcbdfa393e config: Use LiteralSource for some defaults
Using the LiteralSource makes it much easier to find default values, because an IDE reports
the location of a default. With an HCL string they are harder to discover.

Also removes unnecessary mapstructure.Decodes of constant values.
2020-11-20 18:14:17 -05:00
Michael Montgomery ed719c978b Merge branch 'master' into 6074-allow-config-MaxHeaderBytes 2020-11-20 07:43:53 -06:00
Matt Keeler 4bca029be9
Refactor to call non-voting servers read replicas (#9191)
Co-authored-by: Kit Patella <kit@jepsen.io>
2020-11-17 10:53:57 -05:00
Kit Patella 36aaf86647 Merge branch 'mkcp/telemetry/add-all-metric-definitions' of ssh://github.com/hashicorp/consul into mkcp/telemetry/add-all-metric-definitions 2020-11-16 15:26:12 -08:00
Kit Patella 4c30ebbb73 fix some tests that were broken from the TelemetryConfig change 2020-11-16 15:22:36 -08:00
Kit Patella 7ec3ad5b73
linting: sort and group import 2020-11-16 14:17:24 -08:00
Kit Patella 1f0b26c9d3 update runtime_test to handle PrometheusOpts expiry field change 2020-11-16 14:16:12 -08:00
Kit Patella 464d13d80b push prometheus sink definiitons into prometheus.PrometheusOpts 2020-11-16 12:44:47 -08:00
R.B. Boyer a5bd1ba323
agent: return the default ACL policy to callers as a header (#9101)
Header is: X-Consul-Default-ACL-Policy=<allow|deny>

This is of particular utility when fetching matching intentions, as the
fallthrough for a request that doesn't match any intentions is to
enforce using the default acl policy.
2020-11-12 10:38:32 -06:00
Matt Keeler 755fb72994
Switch to using the external autopilot module 2020-11-09 09:22:11 -05:00
R.B. Boyer be9122a8c4
agent: sanitize ui metrics proxy header values on agent/self endpoint (#9104) 2020-11-05 13:25:27 -06:00
R.B. Boyer c8c87ec317
agent: introduce path allow list for requests going through the metrics proxy (#9059)
Added a new option `ui_config.metrics_proxy.path_allowlist`. This defaults to `["/api/v1/query", "/api/v1/query_range"]` when the metrics provider is set to `prometheus`.

Requests that do not use one of the allow-listed paths (via exact match) get a 403 Forbidden response instead.
2020-10-30 16:49:54 -05:00
Michael Montgomery 1c0a46849a Resolves #6074. Adds new option to configure HTTP Server's MaxHeaderBytes with option -http-max-header-bytes
Adds tests for behavior
2020-10-29 12:38:19 -05:00
Daniel Nephin 1dcbfd17c3 health: change the name of UseStreamingBackend config
Remove it from the cache section, and update the docs.
2020-10-23 17:47:01 -04:00
Daniel Nephin dd0e8d42c4
Merge pull request #8825 from hashicorp/streaming/add-config
streaming: add config and docs
2020-10-09 14:33:58 -04:00
Daniel Nephin 195511140f config: add field for enabling streaming in the client
agent: register the new streaming cache-type
2020-10-09 14:11:34 -04:00
Matt Keeler 44017f819e
Remove oss/ent divergence of runtime_test.go (#8903) 2020-10-09 10:45:11 -04:00
Kit Patella 40b9769b1f
Merge pull request #8877 from hashicorp/mkcp/telemetry/consul.api.http
Add flag for disabling 1.9 metrics backwards compatibility and warnings when set to default
2020-10-08 13:22:37 -07:00
Matt Keeler 141eb60f06
Add per-agent reconnect timeouts (#8781)
This allows for client agent to be run in a more stateless manner where they may be abruptly terminated and not expected to come back. If advertising a per-agent reconnect timeout using the advertise_reconnect_timeout configuration when that agent leaves, other agents will wait only that amount of time for the agent to come back before reaping it.

This has the advantageous side effect of causing servers to deregister the node/services/checks for that agent sooner than if the global reconnect_timeout was used.
2020-10-08 15:02:19 -04:00
Daniel Nephin 05df7b18a9 config: add field for enabling streaming RPC endpoint 2020-10-08 12:11:20 -04:00
Kit Patella 328036dd37 add config flag to disable 1.9 metrics backwards compatibility. Add warnings on start and reload on default value 2020-10-07 17:12:52 -07:00