Commit graph

12669 commits

Author SHA1 Message Date
Matt Keeler 8df112526d
Fix some broken code in master
There were several PRs that while all passed CI independently, when they all got merged into the same branch caused compilation errors in test code.

The main changes that caused issues where changing agent/cache.Cache.New to require a concrete options struct instead of a pointer. This broke the cert monitor tests and the catalog_list_services_test.go. Another change was made to unembed the http.Server from the agent.HTTPServer struct. That coupled with another change to add a test to ensure cache rate limiting coming from HTTP requests was working as expected caused compilation failures.
2020-07-28 09:50:10 -04:00
Pierre Souchay 947d8eb039
Added ratelimit to handle throtling cache (#8226)
This implements a solution for #7863

It does:

    Add a new config cache.entry_fetch_rate to limit the number of calls/s for a given cache entry, default value = rate.Inf
    Add cache.entry_fetch_max_burst size of rate limit (default value = 2)

The new configuration now supports the following syntax for instance to allow 1 query every 3s:

    command line HCL: -hcl 'cache = { entry_fetch_rate = 0.333}'
    in JSON

{
  "cache": {
    "entry_fetch_rate": 0.333
  }
}
2020-07-27 23:11:11 +02:00
Blake Covarrubias ec612d7744 docs: Modify website redirects for gateways
Modify the HTTP redirects for the gateway documentation to point to
the updated URL locations. Missed in PR #8195.
2020-07-27 09:27:56 -07:00
Blake Covarrubias 38b41a7a11 docs: Modify network segments limitation callout
Change the callout on the Network Segments page to specify the
limitation applies to versions of Consul prior to version 1.7.3.
2020-07-27 09:10:27 -07:00
John Cowen cae25f4b5a Revert "ui: Use X-Range header/meta to decide whether to reconcile or not"
This reverts commit e9f6361d264a0bece6259ccbb5c438fe463d4c15.
2020-07-27 09:41:16 +00:00
John Cowen 3288d2b7fa Revert "ui: Add X-Range header alongside the HTTP info for services/intentions"
This reverts commit f8771ae3eaf3f75a33a4c3ac6297bc6514c987cf.
2020-07-27 09:40:19 +00:00
John Cowen ee8fdba9f0 ui: Add X-Range header alongside the HTTP info for services/intentions
This header value controls whether ember-data should reconcile its local
store/data or not.
2020-07-27 09:35:39 +00:00
John Cowen 2e51449ace ui: Use X-Range header/meta to decide whether to reconcile or not
Previously we used a `shouldReconcile` method in order to decide whether
a response should trigger a reconciliation of the frontend ember-data
'source of truth' or not. It's a lot nicer/clearer if this 'flag' can be set
alongside the HTTP request information, moreover we almost have the same
functionality in `If-Range`/`Partial Content` HTTP functionality.

Here we partly follow this HTTP semantics but use a custom `X-Range` header
instead.
2020-07-27 09:30:52 +00:00
Blake Covarrubias 8aa4dc9c16 docs: Add redirect for /docs/k8s/connect.html
Redirect /docs/k8s/connect.html to /docs/k8s/connect/overview.
2020-07-24 16:15:47 -07:00
Freddy 642afcd364
Gateway tasks and overview (#8195)
Co-authored-by: Blake Covarrubias <blake@covarrubi.as>
2020-07-24 15:07:36 -06:00
Matt Keeler 3a1058a06b
Move connect root retrieval and cert signing logic out of the RPC endpoints (#8364)
The code now lives on the Server type itself. This was done so that all of this could be shared with auto config certificate signing.
2020-07-24 10:00:51 -04:00
Daniel Nephin 4f634a909f
Merge pull request #8374 from hashicorp/dnephin/dependency-versions
Add script to print available module updates
2020-07-23 16:56:40 -04:00
Daniel Nephin fb73c75411 Add script to print available module updates 2020-07-23 16:13:20 -04:00
Matt Keeler e7d8a02ae8
Move generation of the CA Configuration from the agent code into a method on the RuntimeConfig (#8363)
This allows this to be reused elsewhere.
2020-07-23 16:05:28 -04:00
Kyle Havlovitz cb25e96a63
Merge pull request #8372 from hashicorp/update-go-metrics
vendor: Update github.com/armon/go-metrics to v0.3.3
2020-07-23 12:28:15 -07:00
Kyle Havlovitz 22721d56c8 vendor: Update github.com/armon/go-metrics to v0.3.3 2020-07-23 11:37:33 -07:00
Chris Piraino df35692ef8
Update CHANGELOG.md 2020-07-23 13:15:03 -05:00
Chris Piraino 77b036e6e4
Fix envoy bootstrap logic to not append multiple self_admin clusters (#8371)
Previously, the envoy bootstrap config would blindly copy the self_admin
cluster into the list of static clusters when configuring either
ReadyBindAddr, PrometheusBindAddr, or StatsBindAddr.

Since ingress gateways always configure the ReadyBindAddr property,
users ran into this case much more often than previously.
2020-07-23 13:12:08 -05:00
Daniel Nephin 597dcf2bfb
Merge pull request #8323 from hashicorp/dnephin/add-event-publisher-2
stream: close subscriptions on shutdown
2020-07-23 13:12:50 -04:00
Luke Kysow ebcc925e18
Merge pull request #8368 from hashicorp/lkysow-patch-1
Update terminating-gateways.mdx
2020-07-23 09:45:20 -07:00
Brandon Romano e02baa1e4a
Merge pull request #8370 from hashicorp/br.banner
Update alert banner to HCS GA
2020-07-23 09:37:34 -07:00
Brandon Romano c7c3d7597d Update alert banner to HCS GA 2020-07-23 09:35:41 -07:00
Luke Kysow 41003caadc
Update terminating-gateways.mdx
* fix formatting issue
* use a command that will work with any pod id
2020-07-23 09:19:33 -07:00
Brandon Romano ad504ba196
Merge pull request #8367 from hashicorp/br.cloud-offerings
Adds a cloud offerings section to website
2020-07-23 08:52:57 -07:00
Matt Keeler c3e7d689b7
Refactor the agentpb package (#8362)
First move the whole thing to the top-level proto package name.

Secondly change some things around internally to have sub-packages.
2020-07-23 11:24:20 -04:00
Paul Coignet 9faa0a2a02
Fix tests 2020-07-23 11:04:10 +02:00
Brandon Romano 2b602bbba7 Adds cloud offerings section to website 2020-07-23 01:21:15 -07:00
Daniel Nephin decba06b7d stream: close all subs when EventProcessor is shutdown. 2020-07-22 19:04:10 -04:00
Daniel Nephin e802689bbe stream: fix overallocation in filter
And add tests
2020-07-22 19:04:10 -04:00
Kyle Schochenmaier 3c75847ee2
terminating gw docs (#8300)
* terminating gw docs
Co-authored-by: Rebecca Zanzig <16315901+adilyse@users.noreply.github.com>
Co-authored-by: Blake Covarrubias <blake@covarrubi.as>
2020-07-22 16:02:47 -05:00
Daniel Nephin f64725f7aa state: speed up TestStateStore_ServicesByNodeMeta
Make watchLimit a var so that we can patch it in tests and reduce the time spent creating state.
2020-07-22 16:57:06 -04:00
Daniel Nephin a44ddea9ba state: Use subtests in TestStateStore_ServicesByNodeMeta
These subtests make it much easier to identify the slow part of the test, but they also help enumerate all the different cases which are being tested.
2020-07-22 16:39:09 -04:00
Matt Keeler 2f68d5972a
Update mapstructure to v1.3.3 (#8361)
This was done in preparation for another PR where I was running into https://github.com/mitchellh/mapstructure/issues/202 and implemented a fix for the library.
2020-07-22 15:13:21 -04:00
Geoffrey Grosenbach 68d6a08481
Restore link to /intro docs in main menu (#8356)
The "Overview" link previously went to a summary of Consul features and a
comparison to other products. This commit restores that destination at the
request of Consul PMs.

This commit previously removed the link (it went to `/` instead of `/intro`).

ac612a9cdc (diff-f98b55875118725d1373dd2da36d9ee5)
2020-07-22 10:28:09 -07:00
R.B. Boyer 65e3900a15 update changelog 2020-07-21 15:49:22 -05:00
R.B. Boyer 33f3436e94
gossip: Avoid issue where two unique leave events for the same node could lead to infinite rebroadcast storms (#8343)
bump serf to v0.9.3 to include fix for https://github.com/hashicorp/serf/pull/606
2020-07-21 15:48:10 -05:00
Daniel Nephin 6d3b042872
Merge pull request #7948 from hashicorp/dnephin/buffer-test-logs
testutil: NewLogBuffer - buffer logs until a test fails
2020-07-21 15:21:52 -04:00
Matt Keeler 0baa6b5f0d
Update CHANGELOG.md 2020-07-21 15:09:19 -04:00
Matt Keeler 8ea8a939f0
Merge pull request #8311 from hashicorp/bugfix/auto-encrypt-token-update 2020-07-21 13:15:27 -04:00
Daniel Nephin dbd7e9da39
Merge pull request #8344 from hashicorp/dnephin/fix-flakes-in-stream
stream: handle empty event in TestEventSnapshot
2020-07-21 13:14:35 -04:00
Daniel Nephin 80ff174880 testutil: NewLogBuffer - buffer logs until a test fails
Replaces #7559

Running tests in parallel, with background goroutines, results in test output not being associated with the correct test. `go test` does not make any guarantees about output from goroutines being attributed to the correct test case.

Attaching log output from background goroutines also cause data races.  If the goroutine outlives the test, it will race with the test being marked done. Previously this was noticed as a panic when logging, but with the race detector enabled it is shown as a data race.

The previous solution did not address the problem of correct test attribution because test output could still be hidden when it was associated with a test that did not fail. You would have to look at all of the log output to find the relevant lines. It also made debugging test failures more difficult because each log line was very long.

This commit attempts a new approach. Instead of printing all the logs, only print when a test fails. This should work well when there are a small number of failures, but may not work well when there are many test failures at the same time. In those cases the failures are unlikely a result of a specific test, and the log output is likely less useful.

All of the logs are printed from the test goroutine, so they should be associated with the correct test.

Also removes some test helpers that were not used, or only had a single caller. Packages which expose many functions with similar names can be difficult to use correctly.

Related:
https://github.com/golang/go/issues/38458 (may be fixed in go1.15)
https://github.com/golang/go/issues/38382#issuecomment-612940030
2020-07-21 12:50:40 -04:00
Nathan Lacey a1196feec3
Add certificate disclaimer for TLS encryption (#8316)
Co-authored-by: Freddy <freddygv@users.noreply.github.com>
2020-07-21 10:38:03 -06:00
Matt Keeler 6d94900cd7
Disable background cache refresh for Connect Leaf Certs
The rationale behind removing them is that all of our own code (xDS, builtin connect proxy) use the cache notification mechanism. This ensures that the blocking fetch behind the scenes is always executing. Therefore the only way you might go to get a certificate and have to wait is when 1) the request has never been made for that cert before or 2) you are using the v1/agent/connect/ca/leaf API for retrieving the cert yourself.

In the first case, the refresh change doesn’t alter the behavior. In the second case, it can be mitigated by using blocking queries with that API which just like normal cache notification mechanism will cause the blocking fetch to be initiated and to get leaf certs as soon as needed.

If you are not using blocking queries, or Envoy/xDS, or the builtin connect proxy but are retrieving the certs yourself then the HTTP endpoint might take a little longer to respond.

This also renames the RefreshTimeout field on the register options to QueryTimeout to more accurately reflect that it is used for any type that supports blocking queries.
2020-07-21 12:19:25 -04:00
Matt Keeler 133a6d99f2
Fix issue with changing the agent token causing failure to renew the auto-encrypt certificate
The fallback method would still work but it would get into a state where it would let the certificate expire for 10s before getting a new one. And the new one used the less secure RPC endpoint.

This is also a pretty large refactoring of the auto encrypt code. I was going to write some tests around the certificate monitoring but it was going to be impossible to get a TestAgent configured in such a way that I could write a test that ran in less than an hour or two to exercise the functionality.

Moving the certificate monitoring into its own package will allow for dependency injection and in particular mocking the cache types to control how it hands back certificates and how long those certificates should live. This will allow for exercising the main loop more than would be possible with it coupled so tightly with the Agent.
2020-07-21 12:19:25 -04:00
Daniel Nephin 308815b6bd
Merge pull request #8346 from hashicorp/dnephin/fix-race-in-agent-checks
checks: wait for goroutine to complete (fix go-test-race failures)
2020-07-21 11:57:57 -04:00
duncanKr f026c56891
ui: Table element's have vertical scrollbars (#8296)
Close #8294. Set overflow to hidden for both x and y axis. This prevents the overflow-y defaulting to auto, and creating scrollbars. Given the text overflow is set to ellipsis, this doesn't change the UI functionality.
2020-07-21 09:41:11 -04:00
Ondrej Sika 5538d4ff7a
ui: Fix swapped node & service checks by fixing filter-by & reject-by (#8338) 2020-07-20 19:45:25 -04:00
Daniel Nephin 2d4e5da41e ci: Add jsonfile artifact for go-test-race
So that we can better debug failures
2020-07-20 19:07:58 -04:00
Daniel Nephin f913bed325 checks: wait for goroutine to complete
CheckAlias already had a waitGroup, but the Add() call was happening too late, which was causing a race in tests. The add must happen before the goroutine is started.

CheckHTTP did not have a waitGroup, so I added it to match CheckAlias.

It looks like a lot of the implementation could be shared, and may not need all of channel, waitgroup and bool, but I will leave that refactor for another time.
2020-07-20 18:55:39 -04:00
Daniel Nephin 7599e280de stream: handle empty event in TestEventSnapshot
When the race detector is enabled we see this test fail occasionally. The reordering of execution seems to make it possible for the snapshot splice to happen before any events are published to the topicBuffers.

We can handle this case in the test the same way it is handled by a subscription, by proceeding to the next event.
2020-07-20 18:20:02 -04:00