Commit Graph

3373 Commits

Author SHA1 Message Date
Daniel Nephin 94820e67a8 structs: remove EnterpriseMeta.GetNamespace
I added this recently without realizing that the method already existed and was named
NamespaceOrEmpty. Replace all calls to GetNamespace with NamespaceOrEmpty or NamespaceOrDefault
as appropriate.
2021-03-09 15:17:26 -05:00
Daniel Nephin 848314782e
Merge pull request #9671 from hashicorp/streaming/terminating-gateway-events
state: Add terminating gateway events for streaming
2021-03-09 14:20:21 -05:00
Daniel Nephin 97bc073bd9 state: adjust compare for catalog events
Document that this comparison should roughly match MatchesKey

Only sort by overrideKey or service name, but not both
Add namespace to the sort.

The client side also builds a map of these based on the namespace/node/service key, so the only order
that really matters is the ordering of register/dereigster events.
2021-03-09 14:00:36 -05:00
Daniel Nephin 0d3bb68255 state: handle terminating gateway events properly in snapshot
Refactored out a function that can be used for both the snapshot and stream of events to translate
an event into an appropriate connect event.

Previously terminating gateway events would have used the wrong key in the snapshot, which would have
caused them to be filtered out later on.

Also removed an unused function, and some commented out code.
2021-03-09 14:00:35 -05:00
Kyle Havlovitz de3fba8ef3 Add remaining terminating gateway tests for namespaces
Co-Authored-By: Daniel Nephin <dnephin@hashicorp.com>
2021-03-09 14:00:35 -05:00
Daniel Nephin 38aeb88908 Start to setup enterprise tests for terminating gateway streaming events.
Co-Authored-By: Kyle Havlovitz <kylehav@gmail.com>
2021-03-09 14:00:35 -05:00
Daniel Nephin d0b37f18f0 state: Add support for override of namespace
in MatchesKey
also tests for MatchesKey

Co-Authored-By: Kyle Havlovitz <kylehav@gmail.com>
2021-03-09 14:00:35 -05:00
Daniel Nephin ba59727337 state: update calls to ensureConfigEntryTxn
The EnterpriseMeta paramter was removed after this code was written, but before it merged.

Also the table name constant has changed.
2021-03-09 14:00:35 -05:00
Daniel Nephin 730cc575e6 state: add 2 more test cases for terminate gateway streaming events
Co-Authored-By: Kyle Havlovitz <kylehav@gmail.com>
2021-03-09 14:00:34 -05:00
Kyle Havlovitz eadc8546a9 Added 6 new test cases for terminating gateway events
Co-Authored-By: Daniel Nephin <dnephin@hashicorp.com>
2021-03-09 14:00:34 -05:00
Daniel Nephin 15b0d5f62b state: Add two more tests for connect events with terminating gateways
And expand one test case to cover more.

Co-Authored-By: Kyle Havlovitz <kylehav@gmail.com>
2021-03-09 14:00:34 -05:00
Daniel Nephin abab373b89 state: Include the override key in the sorting of events
Co-Authored-By: Kyle Havlovitz <kylehav@gmail.com>
2021-03-09 14:00:34 -05:00
Kyle Havlovitz f31582624d state: Add terminating gateway events on updating a config entry
Co-Authored-By: Daniel Nephin <dnephin@hashicorp.com>
2021-03-09 14:00:34 -05:00
Daniel Nephin f42a2ca8a3 state: add first terminating catalog catalog event
Health of a terminating gateway instance changes
- Generate an event for creating/destroying this instance of the terminating gateway,
  duplicate it for each affected service

Co-Authored-By: Kyle Havlovitz <kylehav@gmail.com>
2021-03-09 14:00:33 -05:00
Silas Rech f71bba20e7
Fix advertise_addr_wan_ipv6 configuration key 2021-03-09 14:56:44 +01:00
hashicorp-ci da1663bed9 auto-updated agent/uiserver/bindata_assetfs.go from commit 33d038377 2021-03-09 09:35:32 +00:00
freddygv 4bbd495b54 Create new types for service-defaults upstream cfg 2021-03-08 22:10:27 -07:00
hashicorp-ci f0e34dfadb auto-updated agent/uiserver/bindata_assetfs.go from commit 308e5a480 2021-03-08 12:28:15 +00:00
Daniel Nephin 1184ceff9e state: convert nodes.ID to new functional pattern
In preparation for adding other identifiers to the index.
2021-03-05 12:30:40 -05:00
R.B. Boyer 503041f216
xds: default to speaking xDS v3, but allow for v2 to be spoken upon request (#9658)
- Also add support for envoy 1.17.0
2021-02-26 16:23:15 -06:00
Daniel Nephin 4a44cfd676
Merge pull request #9188 from hashicorp/dnephin/more-streaming-tests
Add more streaming tests
2021-02-26 12:36:55 -05:00
Daniel Nephin d7ffd6c27d
Merge pull request #9759 from hashicorp/dnephin/streaming-default-rpc-enabled
streaming: default rpc.enable_streaming to true
2021-02-26 12:08:00 -05:00
Daniel Nephin 4ef9578a07
Merge pull request #9703 from pierresouchay/streaming_tags_and_case_insensitive
Streaming filter tags + case insensitive lookups for Service Names
2021-02-26 12:06:26 -05:00
Daniel Nephin 2cc3282d5d catalog_events: set the right key for connect snapshots
Add a test for catalog_event snapshot on connect topic
2021-02-25 14:30:39 -05:00
Daniel Nephin 85da1af04c consul: Add integration tests of streaming.
Restored from streaming-rpc-final branch.

Co-authored-by: Paul Banks <banks@banksco.de>
2021-02-25 14:30:39 -05:00
Daniel Nephin e8beda4685 state: Add a test for ServiceHealthSnapshot 2021-02-25 14:08:10 -05:00
Daniel Nephin 88bbde56da agent: add a test for streaming in the service health endpoint
Co-authored-by: Paul Banks <banks@banksco.de>
2021-02-25 14:08:10 -05:00
Daniel Nephin af2431793b streaming: default rpc.enable_streaming to true
So that all servers will start the grpc server used by streaming
2021-02-25 14:06:04 -05:00
hashicorp-ci 4ebdbf57d7 auto-updated agent/uiserver/bindata_assetfs.go from commit 779f7f7b6 2021-02-25 09:41:02 +00:00
John Cowen 2d500d24b8
ui: Remove any trailing fullstop/period DNS characters from Gateways UI API (#9752)
Previous to this commit, the API response would include Gateway
Addresses in the form `domain.name.:8080`, which due to the addition of
the port is probably not the expected response.

This commit rightTrims any `.` characters from the end of the domain
before formatting the address to include the port resulting in
`domain.name:8080`
2021-02-25 09:34:47 +00:00
R.B. Boyer 4336d522c1
test: omit envoy golden test files that differ from the latest version (#9807)
Since we currently do no version switching this removes 75% of the PR
noise.

To generate all *.golden files were removed and then I ran:

    go test ./agent/xds -update
2021-02-24 14:04:31 -06:00
hashicorp-ci 8ebffea6a6 auto-updated agent/uiserver/bindata_assetfs.go from commit f9b0e50ca 2021-02-24 09:08:42 +00:00
Yong Wen Chua 677f798443
Update test fixtures 2021-02-24 16:24:32 +08:00
Yong Wen Chua e1820517aa
Auto Rewrite Host Headers for Terminating Gateways
Tries to partially address
https://github.com/hashicorp/consul/issues/8707
2021-02-24 16:23:39 +08:00
hashicorp-ci 26d41f076b auto-updated agent/uiserver/bindata_assetfs.go from commit 8b12d0d09 2021-02-23 09:02:41 +00:00
R.B. Boyer cdc5e99184
xds: remove deprecated usages of xDS (#9602)
Note that this does NOT upgrade to xDS v3. That will come in a future PR.

Additionally:

- Ignored staticcheck warnings about how github.com/golang/protobuf is deprecated.
- Shuffled some agent/xds imports in advance of a later xDS v3 upgrade.
- Remove support for envoy 1.13.x but don't add in 1.17.x yet. We have to wait until the xDS v3 support is added in a follow-up PR.

Fixes #8425
2021-02-22 15:00:15 -06:00
R.B. Boyer b8f228781a
connect: if the token given to the vault provider returns no data avoid a panic (#9806)
Improves #9800
2021-02-22 14:08:49 -06:00
hashicorp-ci 962c5ae214 auto-updated agent/uiserver/bindata_assetfs.go from commit 2c2e9d1ce 2021-02-22 14:58:06 +00:00
Daniel Nephin 8ec1b06f3a
Merge pull request #9683 from hashicorp/dnephin/fix-zombie-service-dereg
local: use agent token to deregister services
2021-02-19 18:56:40 -05:00
Daniel Nephin 76a365d410 local: default to the agent token instead of the user token
When de-registering in anti-entropy sync, when there is no service or
check token.

The agent token will fall back to the default (aka user) token if no agent
token is set, so the existing behaviour still works, but it will prefer
the agent token over the user token if both are set.

ref: https://www.consul.io/docs/agent/options#acl_tokens

The agent token seems more approrpiate in this case, since this is an
"internal operation", not something initiated by the user.
2021-02-19 18:35:08 -05:00
Daniel Nephin 5d478df9b5
Merge pull request #9763 from hashicorp/dnephin/cache-warn-on-error-in-notify
cache: log a warning when Cache.Notify handles an error
2021-02-19 18:30:36 -05:00
Daniel Nephin dd45c4cfe4 state: add a test case for memdb indexers 2021-02-19 17:14:46 -05:00
Daniel Nephin 7e4d693aaa state: support for functional indexers
These new functional indexers provide a few advantages:

1. enterprise differences can be isolated to a single function (the
   indexer function), making code easier to change
2. as a consequence of (1) we no longer need to wrap all the calls to
   Txn operations, making code easier to read.
3. by removing reflection we should increase the performance of all
   operations.

One important change is in making all the function signatures the same.

https://blog.golang.org/errors-are-values

An extra boolean return value for SingleIndexer.FromObject is superfluous.
The error value can indicate when the index value could not be created.
By removing this extra return value we can use the same signature for both
indexer functions.

This has the nice properly of a function being usable for both indexing operations.
2021-02-19 17:14:46 -05:00
Daniel Nephin 88a9bd6d3c state: remove duplicate index on the checks table
By using a new pattern for more specific indexes. This allows us to use
the same index for both service checks and node checks. It removes the
abstraction around memdb.Txn operations, and isolates all of the
enterprise differences in a single place (the indexer).
2021-02-19 17:14:46 -05:00
Daniel Nephin b781fec664 state: remove duplicate function
catalogChecksForNodeService was a duplicate of catalogListServiceChecks
2021-02-19 17:14:46 -05:00
R.B. Boyer e87d2bb24f
xds: only try to create an ipv6 expose checks listener if ipv6 is supported by the kernel (#9765)
Fixes #9311

This only fails if the kernel has ipv6 hard-disabled. It is not sufficient to merely not provide an ipv6 address for a network interface.
2021-02-19 14:38:43 -06:00
hashicorp-ci 0cd20ba653 auto-updated agent/uiserver/bindata_assetfs.go from commit 8263879e6 2021-02-19 16:48:03 +00:00
hashicorp-ci 5622d8b74d auto-updated agent/uiserver/bindata_assetfs.go from commit 7be21a902 2021-02-18 18:37:38 +00:00
hashicorp-ci 7770196315 auto-updated agent/uiserver/bindata_assetfs.go from commit 1507dd8ab 2021-02-17 19:01:18 +00:00
hashicorp-ci 4a8b1c2f0d auto-updated agent/uiserver/bindata_assetfs.go from commit 1b5087f6e 2021-02-17 18:56:34 +00:00
Mark Anderson 1cf6a435dc
Add fields to the /acl/auth-methods endpoint. (#9741)
* A GET of the /acl/auth-method/:name endpoint returns the fields
MaxTokenTTL and TokenLocality, while a LIST (/acl/auth-methods) does
not.

The list command returns a filtered subset of the full set. This is
somewhat deliberate, so that secrets aren't shown, but the TTL and
Locality fields aren't (IMO) security critical, and it is useful for
the front end to be able to show them.

For consistency these changes mirror the 'omit empty' and string
representation choices made for the GET call.

This includes changes to the gRPC and API code in the client.

The new output looks similar to this
curl 'http://localhost:8500/v1/acl/auth-methods' | jq '.'

  {
    "MaxTokenTTL": "8m20s",
    "Name": "minikube-ttl-local2",
    "Type": "kubernetes",
    "Description": "minikube auth method",
    "TokenLocality": "local",
    "CreateIndex": 530,
    "ModifyIndex": 530,
    "Namespace": "default"
  }
]

Signed-off-by: Mark Anderson <manderson@hashicorp.com>

* Add changelog

Signed-off-by: Mark Anderson <manderson@hashicorp.com>
2021-02-17 08:16:57 -08:00
Daniel Nephin 577b3a1370
Merge pull request #9452 from hashicorp/dnephin/config-tests-flags-and-edgecases
config: make TestBuilder_BuildAndValidate_ConfigFlagsAndEdgecases easier to work with
2021-02-16 16:43:49 -05:00
Daniel Nephin d33bc493af
Merge pull request #9720 from hashicorp/dnephin/ent-meta-ergo-1
structs: rename EnterpriseMeta constructor
2021-02-16 15:31:58 -05:00
Daniel Nephin 53c82cee86
Merge pull request #9772 from hashicorp/streamin-fix-bad-cached-snapshot
streaming: fix snapshot cache bug
2021-02-16 15:28:00 -05:00
Daniel Nephin b17967827d
Merge pull request #9728 from hashicorp/dnephin/state-index-table
state: document how index table is used
2021-02-16 15:27:27 -05:00
Daniel Nephin c40d063a0e structs: rename EnterpriseMeta constructor
To match the Go convention.
2021-02-16 14:45:43 -05:00
Daniel Nephin 005702e098 config: rename testCase fields
So that their name better reflects their purpose
2021-02-16 14:40:16 -05:00
Daniel Nephin e6badb3129 config: make TestLoad_IntegrationWithFlags easier to work with
Replace the large table of tests with individual calls to run(). By using
runCase, failure messages will include the line number for the test case, as
well as a line number from the test functions.

Example:
=== FAIL: agent/config TestLoad_IntegrationWithFlags/failing_case (0.01s)
    runtime_test.go:4721: case: failing case
    runtime_test.go:4864: error "data_dir cannot be empty" does not contain "I expected this error"

Previous:
    runtime_test.go:4864: error "data_dir cannot be empty" does not contain "I expected this error"

Without the line number to the testCase data, debugging these tests is
difficult. It is impossible to jump directly to the test case, and
difficult to find the location because of many similarly named cases.
2021-02-16 14:40:16 -05:00
Daniel Nephin db53954a3f move test case iteration to caller
To make the test case logic more obvious
2021-02-16 14:40:16 -05:00
Daniel Nephin 7499e38f2b config: remove test cases for impossible scenarios
AEInterval is overridden by NonUserSource, so there is no way for a user
to set this value. These two cases represented impossible real world
scenarios.

Instead the test is replaced with one that shows that the AEInterval can
not be set by config.

This change allows us to remove the hcltail and jsontail fields from
testCase
2021-02-16 14:40:15 -05:00
Daniel Nephin 5f2c358255 Fix error checking
There were two identical if lines, remove one and use a switch/case so the compiler
can catch this problem.
2021-02-16 14:40:15 -05:00
Daniel Nephin 1ff2525cee config: move testing shims to LoadOpts
This will allow us to eventually port these tests to use config.Load instead of
unexported functions.
2021-02-16 14:40:15 -05:00
Daniel Nephin a877ec79b6 Remove patchActual, does not appear to be necessary
also small rename and fix comment
2021-02-16 14:40:15 -05:00
Daniel Nephin a29b848e3b stream: fix a snapshot cache bug
Previously a snapshot created as part of a resumse-stream request could have incorrectly
cached the newSnapshotToFollow event. This would cause clients to error because they
received an unexpected framing event.
2021-02-16 12:52:23 -05:00
Daniel Nephin 2726c65fbe stream: test the snapshot cache is saved correctly
when the cache entry is created from resuming a stream.
2021-02-16 12:08:43 -05:00
Daniel Nephin e47131bfe6 cache: log a warning when Cache.Notify handles an error
Without these warnings, errors are silently ignored, which can make
debugging problems more challenging.
2021-02-12 13:02:23 -05:00
Daniel Nephin a5a1fb2098
Merge pull request #9758 from hashicorp/dnephin/fix-streaming-bugs
http: error if near is used with streaming
2021-02-12 10:37:29 -05:00
Daniel Nephin 0683964519 streaming: move ServiceTag and NodeMetaFiltering to the cache-entry
So that all the client side filtering is in the same place. Previously
only the bexpr filter was in the cache-entry.

Also makes a small change to the filtering so that instead of rebuilding
slices of items, the filtering can return a bool to determine if the
event payload is saved or not.
2021-02-11 20:20:09 -05:00
Daniel Nephin d8f10deda3
Merge pull request #9727 from mbanikazemi/typos
Correcting the changed function name in comment
2021-02-11 14:46:10 -05:00
Daniel Nephin da62f22aff http: error if near is used with streaming 2021-02-11 14:10:38 -05:00
hashicorp-ci 3b442c77f4 auto-updated agent/uiserver/bindata_assetfs.go from commit 96204a21d 2021-02-11 11:42:07 +00:00
hashicorp-ci 7c29524b95 auto-updated agent/uiserver/bindata_assetfs.go from commit 369637aff 2021-02-11 10:10:21 +00:00
hashicorp-ci cd87ee21f8 auto-updated agent/uiserver/bindata_assetfs.go from commit eda38a0ba 2021-02-11 09:54:29 +00:00
R.B. Boyer 194fb0d144
connect: update supported envoy point releases to 1.16.2, 1.15.3, 1.14.6, 1.13.7 (#9737) 2021-02-10 13:11:15 -06:00
Pierre Souchay 54dbcd0bb9 Do not filter tags unless req.TagFilter is set
Send empty array [] instead of [""] in DNS requests when TagFilter is not set

Do not change case sensitivity of services anymore in `getServiceNodes()` since
cache keys are now case insensitive
2021-02-10 10:36:11 +01:00
Daniel Nephin bd122bb9f5 streaming: double the cache TTL
10 minutes is the default blocking query timeout. Using the same value results in us hitting
the expired cache entry bug frequently. By extending this TTL we at least mitigate the problem.

The underlying bug still needs to be fixed.
2021-02-09 14:36:26 -05:00
Daniel Nephin 3ab546623f submatview: do not reset retry waiter when materializer is reset
The materializer is often reset when an error is received. By resetting
the retryWaiter we effectively never wait. The retryWaiter should only
be reset when we get an event without error. This is done in
Materializer.updateView().
2021-02-09 13:56:50 -05:00
Daniel Nephin 041f96639d api: Use blocking query for health when near is set
Streaming can not be used for these queries because the near query
paramter indicates a specific sort of the results, and that sort
requires data that is not available to the client from the streaming
API.
2021-02-09 13:55:33 -05:00
Pierre Souchay 2c44a731c8 Use lower case for serviceName computation of cache keys 2021-02-09 19:19:40 +01:00
Matt Keeler 19c99dc104
Stop background refresh of cached data for requests that result in ACL not found errors (#9738) 2021-02-09 10:15:53 -05:00
Freddy 5a50b26767
Avoid potential proxycfg/xDS deadlock using non-blocking send 2021-02-08 16:14:06 -07:00
R.B. Boyer 91d9544803
connect: connect CA Roots in the primary datacenter should use a SigningKeyID derived from their local intermediate (#9428)
This fixes an issue where leaf certificates issued in primary
datacenters using Vault as a Connect CA would be reissued very
frequently (every ~20 seconds) because the logic meant to detect root
rotation was errantly triggering.

The hash of the rootCA was being compared against a hash of the
intermediateCA and always failing. This doesn't apply to the Consul
built-in CA provider because there is no intermediate in use in the
primary DC.

This is reminiscent of #6513
2021-02-08 13:18:51 -06:00
Pierre Souchay cec640a1f2 [Streaming] Properly filters node-meta queries on health
This wil fix https://github.com/hashicorp/consul/issues/9730
2021-02-08 17:53:18 +01:00
freddygv a417f88e44 Update comments on avoiding proxycfg deadlock 2021-02-08 09:45:45 -07:00
R.B. Boyer 77424e179a
xds: prevent LDS flaps in mesh gateways due to unstable datacenter lists (#9651)
Also fix a similar issue in Terminating Gateways that was masked by an overzealous test.
2021-02-08 10:19:57 -06:00
Mohammad Banikazemi 45b11c713a Correcting the changed function name in comment
Signed-off-by: Mohammad Banikazemi <mbanikazemi@gmail.com>
2021-02-06 20:23:40 -05:00
freddygv 0a8f2f2105 Retry send after timer fires, in case no updates occur 2021-02-05 18:00:59 -07:00
Daniel Nephin cdda3b9321 state: Use the tableIndex constant 2021-02-05 18:37:45 -05:00
Daniel Nephin de841bd459 state: Document index table
And move the IndexEntry (which is stored in the table) next to the table
schema definition.
2021-02-05 18:37:45 -05:00
R.B. Boyer 05d767b8d6
xds: deduplicate mesh gateway listeners in a stable way (#9650)
In a situation where the mesh gateway is configured to bind to multiple
network interfaces, we use a feature called 'tagged addresses'.
Sometimes an address is duplicated across multiple tags such as 'lan'
and 'lan_ipv4'.

There is code to deduplicate these things when creating envoy listeners,
but that code doesn't ensure that the same tag wins every time. If the
winning tag flaps between xDS discovery requests it will cause the
listener to be drained and replaced.
2021-02-05 16:28:07 -06:00
freddygv 8de6b2590c Make xDS labeling consistent with proxycfg 2021-02-05 15:15:52 -07:00
freddygv 57c29aba5d Update proxycfg logging, labels were already attached 2021-02-05 15:14:49 -07:00
Daniel Nephin 23cfbc8f8d
Merge pull request #9719 from hashicorp/oss/state-store-4
state: remove registerSchema
2021-02-05 14:02:38 -05:00
Daniel Nephin dc70f583d4
Merge pull request #9718 from hashicorp/oss/dnephin/ent-meta-in-state-store-3
state: convert all table name constants to the new prefix pattern
2021-02-05 14:02:07 -05:00
Daniel Nephin eb5d71fd19
Merge pull request #9665 from hashicorp/dnephin/state-store-indexes-2
state: move config-entries table definition to config_entries_schema.go
2021-02-05 14:01:08 -05:00
Daniel Nephin 9beadc578b
Merge pull request #9664 from hashicorp/dnephin/state-store-indexes
state: move ACL schema and index definitions to acl_schema.go
2021-02-05 13:38:31 -05:00
Daniel Nephin b747b27afd state: remove the need for registerSchema
registerSchema creates some indirection which is not necessary in this
case. newDBSchema can call each of the tables.

Enterprise tables can be added from the existing withEnterpriseSchema
shim.
2021-02-05 12:19:56 -05:00
Daniel Nephin 33621706ac state: rename table name constants to use pattern
the 'table' prefix is shorter, and also reads better in queries.
2021-02-05 12:12:19 -05:00
Daniel Nephin 8569295116 state: rename connect constants 2021-02-05 12:12:19 -05:00
Daniel Nephin afdbf2a8ef state: rename table name constants to new pattern
Using Apps Hungarian Notation for these constants makes the memdb queries more readable.
2021-02-05 12:12:18 -05:00
Pierre Souchay c466b08481 Streaming filter tags + case insensitive lookups for Service Names
Will fix:
 * https://github.com/hashicorp/consul/issues/9695
 * https://github.com/hashicorp/consul/issues/9702
2021-02-04 11:00:51 +01:00
Daniel Nephin f929a7117e state: Remove unnecessary entMeta arg to EnsureConfigEntry 2021-02-03 18:10:38 -05:00
freddygv a0be7dcc1d Add trace logs to proxycfg state runner and xds srv 2021-02-02 12:26:38 -07:00
freddygv 0fb96afe31 Avoid potential deadlock using non-blocking send
Deadlock scenario:
    1. Due to scheduling, the state runner sends one snapshot into
    snapCh and then attempts to send a second. The first send succeeds
    because the channel is buffered, but the second blocks.
    2. Separately, Manager.Watch is called by the xDS server after
    getting a discovery request from Envoy. This function acquires the
    manager lock and then blocks on receiving the CurrentSnapshot from
    the state runner.
    3. Separately, there is a Manager goroutine that reads the snapshots
    from the channel in step 1. These reads are done to notify proxy
    watchers, but they require holding the manager lock. This goroutine
    goes to acquire that lock, but can't because it is held by step 2.

Now, the goroutine from step 3 is waiting on the one from step 2 to
release the lock. The goroutine from step 2 won't release the lock until
the goroutine in step 1 advances. But the goroutine in step 1 is waiting
for the one in step 3. Deadlock.

By making this send non-blocking step 1 above can proceed. The coalesce
timer will be reset and a new valid snapshot will be delivered after it
elapses or when one is requested by xDS.
2021-02-02 11:31:14 -07:00
hashicorp-ci 7e7b56e13e auto-updated agent/uiserver/bindata_assetfs.go from commit e0ff7080a 2021-02-02 10:08:48 +00:00
hashicorp-ci d979b67d63 auto-updated agent/uiserver/bindata_assetfs.go from commit 0b7d676dc 2021-02-01 17:55:03 +00:00
hashicorp-ci 083fa1693b auto-updated agent/uiserver/bindata_assetfs.go from commit 3aef5cde2 2021-02-01 17:35:20 +00:00
Kyle Havlovitz 1dee4173c1 connect/ca: Allow ForceWithoutCrossSigning for all providers
This allows setting ForceWithoutCrossSigning when reconfiguring the CA
for any provider, in order to forcibly move to a new root in cases where
the old provider isn't reachable or able to cross-sign for whatever
reason.
2021-01-29 13:38:11 -08:00
hashicorp-ci d156596776 auto-updated agent/uiserver/bindata_assetfs.go from commit 3477b1de7 2021-01-29 16:03:41 +00:00
Daniel Nephin 09425b22a1 state: rename config-entries table const to match new pattern 2021-01-28 20:34:34 -05:00
Daniel Nephin 7d17e20270 state: move config-entries table to new pattern 2021-01-28 20:34:15 -05:00
Daniel Nephin 825b8ade39 state: use indexID
this change was already made to enterprise, so backporting it.
2021-01-28 20:30:08 -05:00
Daniel Nephin 2a262f07fc state: Move ACL schema indexes to match Ent
and use constants for table and index names.
2021-01-28 20:05:09 -05:00
Daniel Nephin 2eea58bcc4
Merge pull request #9302 from hashicorp/dnephin/add-service-3
agent: remove ServiceManager.Start goroutine
2021-01-28 16:59:41 -05:00
Daniel Nephin fc88c28bb9 config: make config.TestLoad_FullConfig use config.Load
This commit makes a number of changes that should make
TestLoad_FullConfig easier to work with, and make the test more like
real world scenarios.

* use separate files in testdata/ dir to store the config source.
  Separate files are much easier to edit because editors can syntax
  highlight json/hcl, and it makes strings easier to find. Previously
  trying to find strings would match strings used in other tests.
* use the exported config.Load interface instead of internal NewBuilder
  and BuildAndValidate.
* remove the tail config overrides, which are only necessary with
  nonZero works.
2021-01-27 17:51:53 -05:00
Daniel Nephin aa42ff5d81 config: Unexport Builder and NewBuilder
This type and constructor are implementation details of config loading.
All callers should use config.Load.
2021-01-27 17:41:53 -05:00
Daniel Nephin 08b2039f5a config: replace calls to config.NewBuilder with config.Load
This is another incremental change to reduce config loading to a single
small interface. All calls to NewBuilder can be replaced with Load.
2021-01-27 17:34:43 -05:00
Daniel Nephin bd545963cc config: improve the interface of Load
This commit reduces the interface to Load() a bit, in preparation for
unexporting NewBuilder and having everything call Load.

The three arguments are reduced to a single argument by moving the other
two into the options struct.

The three return values are reduced to two by moving the RuntimeConfig
and Warnings into a LoadResult struct.
2021-01-27 17:34:43 -05:00
Daniel Nephin 63ba836257
Merge pull request #9252 from hashicorp/dnephin/config-unmethod
config: remove Builder receiver from funcs that dont use it
2021-01-27 17:31:17 -05:00
Matt Keeler 1379b5f7d6
Upgrade raft-autopilot and wait for autopilot it to stop when revoking leadership (#9644)
Fixes: 9626
2021-01-27 11:14:52 -05:00
hashicorp-ci 614c57a9c6 auto-updated agent/uiserver/bindata_assetfs.go from commit 25f989753 2021-01-27 10:47:58 +00:00
Hans Hasselberg 623aab5880
Add flags to support CA generation for Connect (#9585) 2021-01-27 08:52:15 +01:00
hashicorp-ci 773f1c6b21 auto-updated agent/uiserver/bindata_assetfs.go from commit 92f0eb3bd 2021-01-26 18:00:09 +00:00
hashicorp-ci 68f65fd4a9 auto-updated agent/uiserver/bindata_assetfs.go from commit 82a62cd2e 2021-01-26 17:47:18 +00:00
Daniel Nephin 18fcce575a
Merge pull request #9301 from hashicorp/dnephin/add-service-2
agent: reduce AddService 2
2021-01-26 12:01:34 -05:00
R.B. Boyer 5777fa1f59
server: initialize mgw-wanfed to use local gateways more on startup (#9528)
Fixes #9342
2021-01-25 17:30:38 -06:00
Daniel Nephin e1e94ce69c agent: use the new lib/mutex for stateLock
Previously the ServiceManager had to run a separate goroutine so that it could block on a channel
send/receive instead of a lock. Using this mutex with TryLock allows us to cancel the lock when
the serviceConfigWatch is stopped.

Without this change removing the ServiceManager.Start goroutine would not be possible because
when AddService is called it acquires the stateLock. While that lock is held, if there are
existing watches for the service, the old watch will be stopped, and the goroutine holding the
lock will attempt to wait for that watcher goroutine to exit.

If the goroutine is handling an update (serviceConfigWatch.handleUpdate) then it can block on
acquiring the stateLock and deadlock the agent.  With this change the context is cancelled as
and the goroutine will exit instead of waiting on the stateLock.
2021-01-25 18:01:47 -05:00
Daniel Nephin 08e8ed0a7e agent: remove ServiceManager goroutine
The ServiceManager.Start goroutine was used to serialize calls to
agent.addServiceInternal.

All the goroutines which sent events to the channel would block waiting
for a response from that same goroutine, which is effectively the same
as a synchronous call without any channels.

This commit removes the goroutine and channels, and instead calls
addServiceInternal directly. Since all of these goroutines will need to
take the agent.stateLock, the mutex handles the serializing of calls.
2021-01-25 18:01:47 -05:00
Daniel Nephin 081173652c agent: Minor cosmetic changes in ServiceManager
Also use the non-deprecated func in a test
2021-01-25 18:01:47 -05:00
Daniel Nephin 627371bc37 agent: update godoc for AddServiceRequest 2021-01-25 18:01:03 -05:00
Daniel Nephin 121fd18f9a agent: move checkStateSnapshot
Move the field into the struct for addServiceLocked. Also don't require
setting a default value, so that the callers can leave it as nil if they
don't already have a snapshot.
2021-01-25 18:01:03 -05:00
Daniel Nephin 4d0dd9f5ff agent: move two fields off of AddServiceRequest 2021-01-25 18:01:03 -05:00
Daniel Nephin 76ba6c2198 agent: Replace two fields on AddServiceRequest with a func field
The two previous fields were mutually exclusive. They can be represented
with a single function which provides the value.
2021-01-25 18:01:03 -05:00
Daniel Nephin 5e31bdf51a agent: remove serviceRegiration type
Replace with the existing AddServiceRequest struct. These structs are
almost identical. Additionally, the only reason the serviceRegistration
struct existed was to recreate an AddServiceRequest.

By storing and re-using the AddServiceRequest we remove the need to
translate into one type and back to the original type.

We also remove the extra parameters to a function, because those values
are already available from the AddServiceRequest field.

Also a minor optimization to only call tokens.AgentToken() when
necessary. Previous it was being called every time, but the value was
being ignored if the AddServiceRequest had a token.
2021-01-25 18:01:03 -05:00
Daniel Nephin 75b2c55291 agent: remove an a branch in the AddService flow
Handle the decision to use ServiceManager in a single place. Instead of
calling ServiceManager.AddService, then calling back into
addServiceInternal, only call ServiceManager.AddService if we are going
to use it.

This change removes some small duplication and removes a branch from the
AddService flow.
2021-01-25 18:01:03 -05:00
Daniel Nephin 1ce7cdd892 agent: use fields directly, not temp variables
The temprorary variables make it much harder to trace where and how struct
fields are used. If a field is only used a small number of times than
refer to the field directly.
2021-01-25 17:25:04 -05:00
Daniel Nephin 493e987a88 agent: addServiceIternalRequest
Move fields that are only relevant for addServiceInternal onto a new
struct and embed AddServiceRequest.
2021-01-25 17:25:04 -05:00
Daniel Nephin a0b11b4c20 agent: move deprecated AddServiceFromSource to a test file
The method is only used in tests, and only exists for legacy calls.

There was one other package which used this method in tests. Export
the AddServiceRequest and a couple of its fields so the new function can
be used in those tests.
2021-01-25 17:25:03 -05:00
Daniel Nephin de1a80b368 agent: use a single method for Agent.AddService 2021-01-25 17:25:03 -05:00
Daniel Nephin 1c8eca2bfd agent: rename AddService->AddServiceFromSource
In preparation for extracting a single AddService func that accepts a request struct.
2021-01-25 17:25:01 -05:00
Daniel Nephin d7d081f402
Merge pull request #9420 from hashicorp/dnephin/reduce-duplicate-in-catalog-schema
state: reduce interface for Enterprise schema
2021-01-25 17:04:25 -05:00
Chris Boulton 448212060a
connect: add local_request_timeout_ms to configure local_app http timeouts (#9554) 2021-01-25 13:50:00 -06:00
R.B. Boyer 6622185d64
server: use the presense of stored federation state data as a sign that we already activated the federation state feature flag (#9519)
This way we only have to wait for the serf barrier to pass once before
we can make use of federation state APIs Without this patch every
restart needs to re-compute the change.
2021-01-25 13:24:32 -06:00
hashicorp-ci 7c8cfb7a2a auto-updated agent/uiserver/bindata_assetfs.go from commit 09d917618 2021-01-25 19:09:57 +00:00
hashicorp-ci b9d7f04760 auto-updated agent/uiserver/bindata_assetfs.go from commit 88d5e00f9 2021-01-25 18:52:54 +00:00
hashicorp-ci eefa22b79b auto-updated agent/uiserver/bindata_assetfs.go from commit bb9573832 2021-01-25 18:19:10 +00:00
R.B. Boyer e3f5a77ade
server: add OSS stubs supporting validation of source namespaces in service-intentions config entries (#9527) 2021-01-25 11:27:38 -06:00
R.B. Boyer 0247f409a0
server: when wan federating via mesh gateways only do heuristic primary DC bypass on the leader (#9366)
Fixes #9341
2021-01-22 10:03:24 -06:00
hashicorp-ci cae9b96b00 auto-updated agent/uiserver/bindata_assetfs.go from commit 02772e46a 2021-01-20 18:46:55 +00:00
John Cowen d3ecb6d7a0
Fix -ui-content-path without regex (#9569)
* Add templating to inject JSON into an application/json script tag

Plus an external script in order to pick it out and inject the values we
need injecting into ember's environment meta tag.

The UI still uses env style naming (CONSUL_*) but we uses the new style
JSON/golang props behind the scenes.

Co-authored-by: Paul Banks <banks@banksco.de>
2021-01-20 18:40:46 +00:00
hashicorp-ci 30d26fa90e auto-updated agent/uiserver/bindata_assetfs.go from commit ffb6680e7 2021-01-20 17:08:19 +00:00
John Cowen 82a5e6d6fc
api: Ensure the internal/ui/gateway-service-nodes endpoint responds with an array (#9593)
In some circumstances this endpoint will have no results in it (dues to
ACLs, Namespaces, filtering or missing configuration).

This ensures that the response is at least an empty array (`[]`) rather
than `null`
2021-01-20 16:59:02 +00:00
Matt Keeler d871a6f260
Fix flaky test by marking mock expectations as optional (#9596)
These expectations are optional because in a slow CI environment the deadline to cancell the context might occur before the go routine reaches issuing the RPC. Either way we are successfully ensuring context cancellation is working.
2021-01-20 10:58:27 -05:00
hashicorp-ci 884a2d61dc auto-updated agent/uiserver/bindata_assetfs.go from commit 30014ff8f 2021-01-20 15:43:19 +00:00
Freddy 5519051c84
Update topology mapping Refs on all proxy instance deletions (#9589)
* Insert new upstream/downstream mapping to persist new Refs

* Avoid upserting mapping copy if it's a no-op

* Add test with panic repro

* Avoid deleting up/downstreams from inside memdb iterator

* Avoid deleting gateway mappings from inside memdb iterator

* Add CHANGELOG entry

* Tweak changelog entry

Co-authored-by: Paul Banks <banks@banksco.de>
2021-01-20 15:17:26 +00:00
Daniel Nephin 979749d86e state: do not delete from inside an iteration
Deleting from memdb inside an interation can cause a panic from Iterator.Next. This
case is technically safe (for now) because the iterator is using the root radix tree
not a modified one.

However this could break at any time if someone adds an insert or delete to the coordinates table
before this place in the function.

It also sets a bad example, because generally deletes in an interator are not safe. So this
commit uses the pattern we have in other places to move the deletes out of the iteration.
2021-01-19 17:00:07 -05:00
Matt Keeler 2d7a4073e3
Merge pull request #9570 from hashicorp/bugfix/9498 2021-01-19 16:30:04 -05:00
Matt Keeler 2d2ce1fb0c
Ensure that CA initialization does not block leader election.
After fixing that bug I uncovered a couple more:

Fix an issue where we might try to cross sign a cert when we never had a valid root.
Fix a potential issue where reconfiguring the CA could cause either the Vault or AWS PCA CA providers to delete resources that are still required by the new incarnation of the CA.
2021-01-19 15:27:48 -05:00
hashicorp-ci 4b3361fb03 auto-updated agent/uiserver/bindata_assetfs.go from commit be694366a 2021-01-19 15:47:02 +00:00
hashicorp-ci 88ab4cd159 auto-updated agent/uiserver/bindata_assetfs.go from commit 41a4a9f4f 2021-01-19 15:29:55 +00:00
Daniel Nephin 52a1d78e39 state: add a regression test for state store schema
To allow the index to be refactored without accidental changes.

To update the expected value run: 'go test ./agent/consul/state -update'
2021-01-15 18:49:55 -05:00
Daniel Nephin aa21c1ea04 state: reduce interface for Enterprise schema
Using withEnterpriseSchema() we can apply any enterprise schema changes
with a single shim, removing the need to duplicate all of the table
definitions.

Also move all the catalog schemas to a new file to shrink catalog.go a bit.
2021-01-15 18:49:55 -05:00
Daniel Nephin 45c0a71e7e
Merge pull request #8696 from hashicorp/dnephin/fix-load-limits
agent/consul: make Client/Server config reloading more obvious
2021-01-14 17:40:42 -05:00
Daniel Nephin 27a59f86d0
Merge pull request #9436 from hashicorp/dnephin/fix-service-health-req-cache-key
structs: fix caching of ServiceSpecificRequest when ingress=true
2021-01-14 17:26:25 -05:00
Daniel Nephin e8427a48ab agent/consuk: Rename RPCRate -> RPCRateLimit
so that the field name is consistent across config structs.
2021-01-14 17:26:00 -05:00
Daniel Nephin e5320c2db6 agent/consul: make Client/Server config reloading more obvious
I believe this commit also fixes a bug. Previously RPCMaxConnsPerClient was not being re-read from the RuntimeConfig, so passing it to Server.ReloadConfig was never changing the value.

Also improve the test runtime by not doing a lot of unnecessary work.
2021-01-14 17:21:10 -05:00
Daniel Nephin f2b504873a
Merge pull request #9460 from hashicorp/dnephin/fix-data-races
Fix a couple data races in tests
2021-01-14 17:07:01 -05:00
Daniel Nephin ef9d44fdfe structs: fix caching of ServiceSpecificRequest when ingress=true
The field was not being included in the cache info key. This would result in a DNS request for
web.service.consul returning the same result as web.ingress.consul, when those results should
not be the same.
2021-01-14 17:01:40 -05:00
hashicorp-ci 66ec8e5e1f auto-updated agent/uiserver/bindata_assetfs.go from commit 1e30503ec 2021-01-13 09:47:00 +00:00
kevinkengne 18a12d6517
add completeness test for types with CacheInfo method (#9480)
include all fields when fuzzing in tests
split tests by struct type

Ensure the new value for the field is different

fuzzer.Fuzz could produce the same value again in some cases.

Use a custom fuzz function for QueryOptions. That type is an embedded struct in the request types
but only one of the fields is important to include in the cache key.

Move enterpriseMetaField to an oss file so that we can change it in enterprise.
2021-01-12 19:45:46 -05:00
Chris Piraino baad708929
Fix bug in usage metrics when multiple service instances are changed in a single transaction (#9440)
* Fix bug in usage metrics that caused a negative count to occur

There were a couple of instances were usage metrics would do the wrong
thing and result in incorrect counts, causing the count to attempt to
decrement below zero and return an error. The usage metrics did not
account for various places where a single transaction could
delete/update/add multiple service instances at once.

We also remove the error when attempting to decrement below zero, and
instead just make sure we do not accidentally underflow the unsigned
integer. This is a more graceful failure than returning an error and not
allowing a transaction to commit.

* Add changelog
2021-01-12 15:31:47 -06:00
hashicorp-ci 5b92255cf5 auto-updated agent/uiserver/bindata_assetfs.go from commit b86eea4fb 2021-01-12 14:57:52 +00:00
Daniel Nephin 51464763f4 config: remove Builder receiver from funcs that dont use it
This change allows us to re-use these functions in other places without the Builder, and makes it
more explicit about which functions can warn/error and which can not.
2021-01-11 17:41:58 -05:00
Daniel Nephin 78b02663e9 config: Use golden for TestRuntimeConfig_Sanitize
A golden file makes the expected value easier to work with. This change also
removes a number of shims for enterprise and replaces them with a single one
for the golden filename.
2021-01-11 14:34:03 -05:00
Pierre Souchay 70d2da7582
Display a warning when rpc.enable_streaming = true is set on a client (#9530)
* Display a warning when rpc.enable_streaming = true is set on a client

This option has no effect when running as an agent

* Added warning when server starts with use_streaming_backend but without rpc.enable_streaming

* Added unit test
2021-01-08 15:23:23 -05:00
Chris Piraino 2eac571276
Log replication warnings when no error suppression is defined (#9320)
* Log replication warnings when no error suppression is defined

* Add changelog file
2021-01-08 14:03:06 -06:00
Daniel Nephin 6094b73d89
Merge pull request #9250 from hashicorp/dnephin/config-tags-2
config: remove unnecessary json/hcl struct tags
2021-01-08 14:08:34 -05:00
Daniel Nephin d5bdc2f539
Merge pull request #9249 from hashicorp/dnephin/config-tags
config: use fields to detect enterprise-only settings
2021-01-07 19:49:29 -05:00
Daniel Nephin f3b30a7ed6 config: remove unnecessary json/hcl struct tags
Using find/replace with regex

        json:"[^"]+"
        hcl:"[^"]+"
2021-01-07 19:42:56 -05:00
Daniel Nephin ee00d7668c config: Use config fields to warn about enterprise settings
It is no safe to assumes that the mapstructure keys will contain all the keys because some config can be specified
with command line flags or literals.

This change allows us to remove the json marshal/unmarshal cycle for command line flags, which will allow
us to remove all of the hcl/json struct tags on config fields.
2021-01-07 19:31:39 -05:00
Daniel Nephin 0679a23921
Merge pull request #9251 from hashicorp/dnephin/config-cleanup-flags-and-opts
config: move testing shims and cleanup some defaults
2021-01-07 19:11:22 -05:00
Daniel Nephin 942ffa7e02 config: use LiteralConfig for flag values 2021-01-07 18:57:53 -05:00
Daniel Nephin 2d0f33591e config: remove t.Parallel and some unused fields from tests
Use go-cmp for better diffs in tests
2021-01-07 18:57:52 -05:00
Daniel Nephin 31b1addd9e structs: add tests for String() methods
To show that printing one of these IDs works properly now that the String() method
receiver is no longer a pointer.
2021-01-07 18:47:38 -05:00
Daniel Nephin 45f0afcbf4 structs: Fix printing of IDs
These types are used as values (not pointers) in other structs. Using a pointer receiver causes
problems when the value is printed. fmt will not call the String method if it is passed a value
and the String method has a pointer receiver. By using a value receiver the correct string is printed.

Also remove some unused methods.
2021-01-07 18:47:38 -05:00
Daniel Nephin bcfb444a77 Remove an unnecessary else 2021-01-07 18:13:49 -05:00
Daniel Nephin f6543b1651 xds: remove Server.Initialize
Requiring a call to initialize to set a single field is not really substantially different
from having to set that field to a value.
2021-01-07 18:13:48 -05:00
Daniel Nephin bbf1a116f6 xds: Fix data race
TestEnvoy.Close used e.stream.recvCh == nil to indicate the channel had already
been closed, so that TestEnvoy.Close can be called multiple times. The recvCh
was not protected by a lock, so setting it to nil caused a data race with any
goroutine trying to read from the channel.

Instead set the stream to nil. The stream is guarded by a lock, so it does not race.

This change allows us to test the agent/xds package using -race.
2021-01-07 18:13:48 -05:00
Daniel Nephin de226f26e4 xds: Pass in logger
small cleanup in tests
2021-01-07 18:13:48 -05:00
hashicorp-ci 7c5ca27fdd auto-updated agent/uiserver/bindata_assetfs.go from commit e893ba7ea 2021-01-07 19:09:58 +00:00
Daniel Nephin 27c38bfebb
Merge pull request #9213 from hashicorp/dnephin/resolve-tokens-take-2
acl: Remove some unused things and document delegate method
2021-01-06 18:51:51 -05:00
Pierre Souchay c43888c064 Added testing of GRPC with TLS combinations
This ensures that https://github.com/hashicorp/consul/issues/9474 will
not reproduce.
2021-01-06 22:20:23 +01:00
Pierre Souchay 542852786c [Streaming][bugfix] handle TLS signalisation when TLS is disabled on client side
Tnis is an alternative to https://github.com/hashicorp/consul/pull/9494
2021-01-06 17:24:58 +01:00
R.B. Boyer db62541676
acl: use the presence of a management policy in the state store as a sign that we already migrated to v2 acls (#9505)
This way we only have to wait for the serf barrier to pass once before
we can upgrade to v2 acls. Without this patch every restart needs to
re-compute the change, and potentially if a stray older node joins after
a migration it might regress back to v1 mode which would be problematic.
2021-01-05 17:04:27 -06:00
hashicorp-ci 8fc77877be auto-updated agent/uiserver/bindata_assetfs.go from commit 1304dc882 2021-01-05 17:47:53 +00:00
Daniel Nephin 3140c0a343
Merge pull request #9067 from naemono/6074-allow-config-MaxHeaderBytes
Adds option to configure HTTP Server's MaxHeaderBytes
2021-01-05 12:28:27 -05:00
hashicorp-ci e64b90c604 auto-updated agent/uiserver/bindata_assetfs.go from commit a42e844cc 2021-01-05 17:09:19 +00:00
hashicorp-ci 988609fb61 auto-updated agent/uiserver/bindata_assetfs.go from commit 17438020f 2021-01-05 10:11:12 +00:00
Michael Montgomery 5a70c2c7e8 Remove unneeded test 2021-01-04 19:47:13 -06:00
Matt Keeler 3a79b559f9
Special case the error returned when we have a Raft leader but are not tracking it in the ServerLookup (#9487)
This can happen when one other node in the cluster such as a client is unable to communicate with the leader server and sees it as failed. When that happens its failing status eventually gets propagated to the other servers in the cluster and eventually this can result in RPCs returning “No cluster leader” error.

That error is misleading and unhelpful for determing the root cause of the issue as its not raft stability but rather and client -> server networking issue. Therefore this commit will add a new error that will be returned in that case to differentiate between the two cases.
2021-01-04 14:05:23 -05:00
hashicorp-ci 1e58b31098 auto-updated agent/uiserver/bindata_assetfs.go from commit 8c9d5ecc2 2021-01-04 18:36:22 +00:00
R.B. Boyer 42dea6f01e
server: deletions of intentions by name using the intention API is now idempotent (#9278)
Restoring a behavior inadvertently changed while fixing #9254
2021-01-04 11:27:00 -06:00
hashicorp-ci 091963c7ea auto-updated agent/uiserver/bindata_assetfs.go from commit 8c0473a62 2021-01-04 16:52:35 +00:00
Michael Montgomery a1748aa2cb Merge branch 'master' into 6074-allow-config-MaxHeaderBytes 2020-12-30 14:14:05 -06:00
Michael Montgomery 519f537b8b Fixed failing tests
Removed use of `NewTestAgent`, per review comment
Removed CLI flag, per review comment
Updated website documentation
Added changelog entry
2020-12-30 14:09:50 -06:00
Daniel Nephin 088831c91e Maybe fix another data race in a test 2020-12-22 18:53:54 -05:00
Daniel Nephin d0f2eca8de Fix one race caused by t.Parallel 2020-12-22 18:27:18 -05:00
hashicorp-ci 5786746ed5 auto-updated agent/uiserver/bindata_assetfs.go from commit 99f102705 2020-12-18 10:43:59 +00:00
hashicorp-ci 098ca1f567 auto-updated agent/uiserver/bindata_assetfs.go from commit 25d6a1277 2020-12-18 09:07:19 +00:00
hashicorp-ci 87b547cfb3 auto-updated agent/uiserver/bindata_assetfs.go from commit c7d917777 2020-12-17 19:01:42 +00:00
hashicorp-ci d302cf769c auto-updated agent/uiserver/bindata_assetfs.go from commit 2e3a66efb 2020-12-17 16:39:34 +00:00
hashicorp-ci 325bca338b auto-updated agent/uiserver/bindata_assetfs.go from commit 635cf4dc9 2020-12-17 16:08:30 +00:00
hashicorp-ci 33fe41ac33 auto-updated agent/uiserver/bindata_assetfs.go from commit 66cc91c69 2020-12-17 15:30:54 +00:00
hashicorp-ci 9ad738b0bf auto-updated agent/uiserver/bindata_assetfs.go from commit 0ca54c608 2020-12-16 16:46:08 +00:00
hashicorp-ci d600dc85ea auto-updated agent/uiserver/bindata_assetfs.go from commit 4404b4f44 2020-12-16 09:23:22 +00:00
hashicorp-ci fff6a07022 auto-updated agent/uiserver/bindata_assetfs.go from commit a919b60f5 2020-12-15 19:36:10 +00:00
hashicorp-ci b7c34f4b71 auto-updated agent/uiserver/bindata_assetfs.go from commit e921b3cf9 2020-12-15 18:33:36 +00:00
hashicorp-ci d0dffb1697 auto-updated agent/uiserver/bindata_assetfs.go from commit 5e150d7f0 2020-12-15 18:19:30 +00:00
John Cowen d3d4c1452a
api: Ensure the internal/ui/service endpoint responds with an array (#9397)
In some circumstances this endpoint will have no results in it (dues to
ACLs, Namespaces or filtering).

This ensures that the response is at least an empty array (`[]`) rather
than `null`
2020-12-15 16:52:00 +00:00
hashicorp-ci 50bad6e642 auto-updated agent/uiserver/bindata_assetfs.go from commit 9f0787197 2020-12-15 16:37:21 +00:00
hashicorp-ci cc3ebef5df auto-updated agent/uiserver/bindata_assetfs.go from commit f111d6b3e 2020-12-15 15:40:26 +00:00
hashicorp-ci de315911b4 auto-updated agent/uiserver/bindata_assetfs.go from commit 14d043e5f 2020-12-14 15:33:47 +00:00
hashicorp-ci 88b34b3996 auto-updated agent/uiserver/bindata_assetfs.go from commit 4e419b9b3 2020-12-14 14:31:06 +00:00
Daniel Nephin b58401480b http: Check HTTPUseCache in a single place
HTTPUseCache is only used is a gate for allowing QueryOptions.UseCache to be enabled. By
moving it to the place where the query options are set, this behaviour is more obvious.

Also remove parseInternal which was an alias for parse.
2020-12-11 14:03:47 -05:00
Daniel Nephin c66a63275f
Merge pull request #9340 from hashicorp/dnephin/skip-slow-tests-with-short
testing: skip slow tests with -short
2020-12-11 13:33:44 -05:00
hashicorp-ci 91c244a830 auto-updated agent/uiserver/bindata_assetfs.go from commit 514270a41 2020-12-11 11:48:26 +00:00
hashicorp-ci 71cd7c6a16 auto-updated agent/uiserver/bindata_assetfs.go from commit 0f7c909f1 2020-12-11 09:44:56 +00:00
hashicorp-ci f827deb8a7 auto-updated agent/uiserver/bindata_assetfs.go from commit 21e5a8f0f 2020-12-11 09:38:09 +00:00
R.B. Boyer f9dcaf7f6b
acl: global tokens created by auth methods now correctly replicate to secondary datacenters (#9351)
Previously the tokens would fail to insert into the secondary's state
store because the AuthMethod field of the ACLToken did not point to a
known auth method from the primary.
2020-12-09 15:22:29 -06:00
hashicorp-ci 6e08ef263e auto-updated agent/uiserver/bindata_assetfs.go from commit 9d8131907 2020-12-09 19:17:30 +00:00
hashicorp-ci 52ca0f2a2b auto-updated agent/uiserver/bindata_assetfs.go from commit a78566e2d 2020-12-09 18:54:02 +00:00
hashicorp-ci bfaf17fa05 auto-updated agent/uiserver/bindata_assetfs.go from commit d6cb2b0d7 2020-12-09 18:45:39 +00:00
Kenia 8bda36c9f4
Create consul version metric with version label (#9350)
* create consul version metric with version label

* agent/agent.go: add pre-release Version as well as label

Co-Authored-By: Radha13 <kumari.radha3@gmail.com>

* verion and pre-release version labels.

* hyphen/- breaks prometheus

* Add Prometheus gauge defintion for version metric

* Add new metric to telemetry docs

Co-authored-by: Radha Kumari <kumari.radha3@gmail.com>
Co-authored-by: Aestek <thib.gilles@gmail.com>
Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>
2020-12-09 09:16:53 -05:00
hashicorp-ci 3f2199d323 auto-updated agent/uiserver/bindata_assetfs.go from commit 27c74f714 2020-12-09 13:12:32 +00:00
hashicorp-ci cdfae9ce63 auto-updated agent/uiserver/bindata_assetfs.go from commit 613be01f4 2020-12-09 09:29:17 +00:00
hashicorp-ci 37613345f2 auto-updated agent/uiserver/bindata_assetfs.go from commit db5283ee2 2020-12-08 15:53:31 +00:00
hashicorp-ci 535ef33bac auto-updated agent/uiserver/bindata_assetfs.go from commit 3be03029f 2020-12-08 09:32:32 +00:00
Daniel Nephin ef0999547a testing: skip slow tests with -short
Add a skip condition to all tests slower than 100ms.

This change was made using `gotestsum tool slowest` with data from the
last 3 CI runs of master.
See https://github.com/gotestyourself/gotestsum#finding-and-skipping-slow-tests

With this change:

```
$ time go test -count=1 -short ./agent
ok      github.com/hashicorp/consul/agent       0.743s

real    0m4.791s

$ time go test -count=1 -short ./agent/consul
ok      github.com/hashicorp/consul/agent/consul        4.229s

real    0m8.769s
```
2020-12-07 13:42:55 -05:00
hashicorp-ci d4478782ae auto-updated agent/uiserver/bindata_assetfs.go from commit 4dfa7622d 2020-12-07 09:24:00 +00:00
hashicorp-ci b734cf54ed auto-updated agent/uiserver/bindata_assetfs.go from commit adbd6c0c8 2020-12-03 09:19:50 +00:00
Kyle Havlovitz 57210a59c3 connect: Fix a case where the active root would get unset even when there wasn't a new one 2020-12-02 11:42:23 -08:00
hashicorp-ci 4ee15914b0 auto-updated agent/uiserver/bindata_assetfs.go from commit e23b5b003 2020-12-02 15:53:16 +00:00
hashicorp-ci a814338374 auto-updated agent/uiserver/bindata_assetfs.go from commit 9ac7bc180 2020-12-02 15:46:59 +00:00
hashicorp-ci 297210ad93 auto-updated agent/uiserver/bindata_assetfs.go from commit a5b9ada9a 2020-12-02 09:49:40 +00:00
hashicorp-ci af9687e0b8 auto-updated agent/uiserver/bindata_assetfs.go from commit cf38309f6 2020-12-01 15:49:06 +00:00
Kyle Havlovitz 91d5d6c586
Merge pull request #9009 from hashicorp/update-secondary-ca
connect: Fix an issue with updating CA config in a secondary datacenter
2020-11-30 14:49:28 -08:00
Kyle Havlovitz c5167cf9c4 Use a buffered channel for CA intermediate renew func 2020-11-30 14:37:24 -08:00
hashicorp-ci 149e1e5f13 auto-updated agent/uiserver/bindata_assetfs.go from commit afe0f2614 2020-11-30 18:47:37 +00:00
hashicorp-ci 45f7de452f auto-updated agent/uiserver/bindata_assetfs.go from commit b5abbf122 2020-11-30 17:33:21 +00:00
hashicorp-ci 58797598dc auto-updated agent/uiserver/bindata_assetfs.go from commit d1ebe8c14 2020-11-30 17:27:35 +00:00
hashicorp-ci c03baa7b57 auto-updated agent/uiserver/bindata_assetfs.go from commit f46ef3e3f 2020-11-30 17:07:25 +00:00
hashicorp-ci 4801228104 auto-updated agent/uiserver/bindata_assetfs.go from commit a59a2f860 2020-11-30 16:57:34 +00:00
hashicorp-ci 2fd62ba8de auto-updated agent/uiserver/bindata_assetfs.go from commit 9cf30e74e 2020-11-30 15:09:43 +00:00
Daniel Nephin 17a86be022
Merge pull request #9284 from hashicorp/dnephin/agent-service-register
local: mark service as InSync when added to local agent state
2020-11-27 15:49:55 -05:00
Daniel Nephin 4c5fab6e00 local: mark service and checks as InSync when added
If the existing service and checks are the same as the new registration.
2020-11-27 15:31:12 -05:00
Hans Hasselberg 8c5c6e77ec
fix serf_wan documentation (#9289)
WAN config is different than LAN config, source of truth is
f72d2042a8/config.go (L315-L326)
and now the docs are correct.
2020-11-27 20:49:43 +01:00
hashicorp-ci b6e469e1e9 auto-updated agent/uiserver/bindata_assetfs.go from commit 408174f3b 2020-11-27 15:45:17 +00:00
Daniel Nephin 813f0d552d
Merge pull request #9247 from pierresouchay/streaming_predictible_order_for_health
[Streaming] Predictable order for results of /health/service/:serviceName to mimic memdb
2020-11-25 15:53:18 -05:00
Pierre Souchay 09673426e3 Applied suggestions from @dnephin
* Renamed `cachedHealResultSorter` into `sortCheckServiceNodes`
* Use `<` instead of `strings.Compare`
* Single line comparison in unit test
2020-11-25 21:40:51 +01:00
R.B. Boyer 6d6b6c15c6
server: fix panic when deleting a non existent intention (#9254)
* server: fix panic when deleting a non existent intention

* add changelog

* Always return an error when deleting non-existent ixn

Co-authored-by: freddygv <gh@freddygv.xyz>
2020-11-24 13:44:20 -05:00
hashicorp-ci 293360339b auto-updated agent/uiserver/bindata_assetfs.go from commit 6f8b5acbe 2020-11-24 17:51:46 +00:00
hashicorp-ci 4039a19ed3 auto-updated agent/uiserver/bindata_assetfs.go from commit 9c3c7bcf3 2020-11-24 14:38:24 +00:00
Hans Hasselberg 25f9e232af add missing descriptions for metrics 2020-11-23 22:06:30 +01:00
Kit Patella 7a8844ccce add entries for missing fsm operations and mark duplicated metrics prefixes as deprecated 2020-11-23 12:42:51 -08:00
Daniel Nephin 1987a1eca0 config: remove unused const 2020-11-20 19:17:12 -05:00
Kyle Havlovitz a01f853aa5 Clean up the logic in persistNewRootAndConfig 2020-11-20 15:54:44 -08:00
Daniel Nephin 8e783fb37b config: move testing shims to BuilderOpts
And remove the devMode field from builder.

This change helps make the Builder state more explicit by moving inputs to the BuilderOps struct,
leaving only fields that can change during Builder.Build on the Builder struct.
2020-11-20 18:31:10 -05:00
Daniel Nephin fcbdfa393e config: Use LiteralSource for some defaults
Using the LiteralSource makes it much easier to find default values, because an IDE reports
the location of a default. With an HCL string they are harder to discover.

Also removes unnecessary mapstructure.Decodes of constant values.
2020-11-20 18:14:17 -05:00
Kit Patella 3ea27d75e4
Merge pull request #9245 from hashicorp/telemetry/fix-missing-and-stale-docs
Telemetry/fix missing and stale docs
2020-11-20 12:54:29 -08:00
Pierre Souchay 9239df6dbd [Streaming] Predictable order for results of /health/service/:serviceName to mimic memdb
This ensures the result is consitent with/witout streaming

Will partially fix #9239
2020-11-20 16:23:35 +01:00
Michael Montgomery ed719c978b Merge branch 'master' into 6074-allow-config-MaxHeaderBytes 2020-11-20 07:43:53 -06:00
Kyle Havlovitz 26a9c985c5 Add CA server delegate interface for testing 2020-11-19 20:08:06 -08:00
Kit Patella 4ad076207e add telemetry and definition help entries for missing catalog and acl metrics 2020-11-19 13:29:44 -08:00
R.B. Boyer 7bcbc59dea
command: when generating envoy bootstrap configs use the datacenter returned from the agent services endpoint (#9229)
Fixes #9215
2020-11-19 15:27:31 -06:00
Kit Patella 46205bbf27 remove stale entries and rename/define acl.resolveToken 2020-11-19 13:06:28 -08:00
hashicorp-ci 22a0ab69ae auto-updated agent/uiserver/bindata_assetfs.go from commit d913af2bb 2020-11-19 18:45:01 +00:00
Freddy e4e306210a
Require operator:write to get Connect CA config (#9240)
A vulnerability was identified in Consul and Consul Enterprise (“Consul”) such that operators with `operator:read` ACL permissions are able to read the Consul Connect CA configuration when explicitly configured with the `/v1/connect/ca/configuration` endpoint, including the private key. This allows the user to effectively privilege escalate by enabling the ability to mint certificates for any Consul Connect services. This would potentially allow them to masquerade (receive/send traffic) as any service in the mesh.

--

This PR increases the permissions required to read the Connect CA's private key when it was configured via the `/connect/ca/configuration` endpoint. They are now `operator:write`.
2020-11-19 10:14:48 -07:00
hashicorp-ci 293ba9e0b5 auto-updated agent/uiserver/bindata_assetfs.go from commit 687ce1f9c 2020-11-19 16:13:04 +00:00
Daniel Nephin 35c5f83ea3
Merge pull request #9224 from hashicorp/dnephin/fix-multiple-http-listeners
agent: fix bug with multiple listeners
2020-11-18 16:52:29 -05:00
Daniel Nephin 8647483605 Use freeport
To prevent other tests which already use freeport from flaking when port 0 steals their reserved port.
2020-11-18 16:07:34 -05:00
hashicorp-ci 75a1727b31 auto-updated agent/uiserver/bindata_assetfs.go from commit 591a96d5b 2020-11-18 19:07:25 +00:00
hashicorp-ci fc07c63974 auto-updated agent/uiserver/bindata_assetfs.go from commit 1edef424a 2020-11-18 19:00:19 +00:00
Daniel Nephin fed2a61dfc agent: fix bug with multiple listeners
Previously the listener was being passed to a closure in a loop without
capturing the loop variable. The result is only the last listener is
used, so the http/https servers only listen on one address.

This problem is fixed by capturing the variable by passing it into a
function.
2020-11-18 13:03:29 -05:00
hashicorp-ci 393d83dfa3 auto-updated agent/uiserver/bindata_assetfs.go from commit 664f1d9aa 2020-11-18 11:17:06 +00:00
Kyle Havlovitz c8d4a40a87 connect: update some function comments in CA manager 2020-11-17 16:00:19 -08:00
Daniel Nephin b9306d8827 acl: remove a test-only method 2020-11-17 18:16:34 -05:00
Daniel Nephin 9e7c8dd19d Remove two unused delegate methods 2020-11-17 18:16:26 -05:00
Daniel Nephin d9af48afce
Merge pull request #9160 from hashicorp/dnephin/go-test-race-in-to-out-list
ci: change go-test-race package list to exclude list
2020-11-17 13:13:38 -05:00
Matt Keeler 4bca029be9
Refactor to call non-voting servers read replicas (#9191)
Co-authored-by: Kit Patella <kit@jepsen.io>
2020-11-17 10:53:57 -05:00
Kit Patella 4dfcdbab26
Merge pull request #9198 from hashicorp/mkcp/telemetry/add-all-metric-definitions
Add metric definitions for all metrics known at Consul start
2020-11-16 15:54:50 -08:00
hashicorp-ci 95fa102195 auto-updated agent/uiserver/bindata_assetfs.go from commit fe728855e 2020-11-16 23:41:31 +00:00
Freddy 2763833d32
Add DC and NS support for Envoy metrics (#9207)
This PR updates the tags that we generate for Envoy stats.

Several of these come with breaking changes, since we can't keep two stats prefixes for a filter.
2020-11-16 16:37:19 -07:00
Kit Patella 36aaf86647 Merge branch 'mkcp/telemetry/add-all-metric-definitions' of ssh://github.com/hashicorp/consul into mkcp/telemetry/add-all-metric-definitions 2020-11-16 15:26:12 -08:00
Kit Patella 4c30ebbb73 fix some tests that were broken from the TelemetryConfig change 2020-11-16 15:22:36 -08:00
Kit Patella 7ec3ad5b73
linting: sort and group import 2020-11-16 14:17:24 -08:00
Kit Patella 1f0b26c9d3 update runtime_test to handle PrometheusOpts expiry field change 2020-11-16 14:16:12 -08:00
Matt Keeler 197a37a860
Prevent panic if autopilot health is requested prior to leader establishment finishing. (#9204) 2020-11-16 17:08:17 -05:00
Kit Patella 6290be054a use the MetricsPrefix to set the service name and provide as slice literal to avoid bugs from append modifying its first arg 2020-11-16 14:01:12 -08:00
Kit Patella 464d13d80b push prometheus sink definiitons into prometheus.PrometheusOpts 2020-11-16 12:44:47 -08:00
Daniel Nephin de88ceed1c
Merge pull request #9114 from hashicorp/dnephin/filtering-in-stream
stream: improve naming of Payload methods
2020-11-16 14:20:07 -05:00
Kit Patella 0b18f5612e trim help strings to save a few bytes 2020-11-16 11:02:11 -08:00
Kit Patella 374748dafc merge master 2020-11-16 10:46:53 -08:00
hashicorp-ci 42641671b3 auto-updated agent/uiserver/bindata_assetfs.go from commit 959974e96 2020-11-16 15:27:40 +00:00
Kit Patella af719981f3 finish adding static server metrics 2020-11-13 16:26:08 -08:00
Kyle Havlovitz 0a86533e20 Reorganize some CA manager code for correctness/readability 2020-11-13 14:46:01 -08:00
Kyle Havlovitz 5de81c1375 connect: Add CAManager for synchronizing CA operations 2020-11-13 14:33:44 -08:00
Kyle Havlovitz 0b4876f906 connect: Add logic for updating secondary DC intermediate on config set 2020-11-13 14:33:44 -08:00
R.B. Boyer db1184c094
server: intentions CRUD requires connect to be enabled (#9194)
Fixes #9123
2020-11-13 16:19:12 -06:00
Kit Patella b486c1bce8 add the service name in the agent rather than in the definitions themselves 2020-11-13 13:18:04 -08:00
R.B. Boyer e323014faf
server: remove config entry CAS in legacy intention API bridge code (#9151)
Change so line-item intention edits via the API are handled via the state store instead of via CAS operations.

Fixes #9143
2020-11-13 14:42:21 -06:00
R.B. Boyer 6300abed18
server: skip deleted and deleting namespaces when migrating intentions to config entries (#9186) 2020-11-13 13:56:41 -06:00
Mike Morris a343365da7
ci: update to Go 1.15.4 and alpine:3.12 (#9036)
* ci: stop building darwin/386 binaries

Go 1.15 drops support for 32-bit binaries on Darwin https://golang.org/doc/go1.15#darwin

* tls: ConnectionState::NegotiatedProtocolIsMutual is deprecated in Go 1.15, this value is always true

* correct error messages that changed slightly

* Completely regenerate some TLS test data

Co-authored-by: R.B. Boyer <rb@hashicorp.com>
2020-11-13 13:02:59 -05:00
hashicorp-ci 45a8bc9472 auto-updated agent/uiserver/bindata_assetfs.go from commit 1059a51a3 2020-11-13 16:00:39 +00:00
hashicorp-ci b8d6e195ed auto-updated agent/uiserver/bindata_assetfs.go from commit 78b704be8 2020-11-13 15:44:14 +00:00
R.B. Boyer 758384893d
server: break up Intention.Apply monolithic method (#9007)
The Intention.Apply RPC is quite large, so this PR attempts to break it down into smaller functions and dissolves the pre-config-entry approach to the breakdown as it only confused things.
2020-11-13 09:15:39 -06:00
Kit Patella 9533372ded first pass on agent-configured prometheusDefs and adding defs for every consul metric 2020-11-12 18:12:12 -08:00
Daniel Nephin 70093be98c
Merge pull request #9162 from hashicorp/dnephin/fix-grpc-metrics
grpc: fix metrics
2020-11-12 17:03:01 -05:00
hashicorp-ci dc2cb412b8 auto-updated agent/uiserver/bindata_assetfs.go from commit 6b2970402 2020-11-12 18:49:48 +00:00
R.B. Boyer a5bd1ba323
agent: return the default ACL policy to callers as a header (#9101)
Header is: X-Consul-Default-ACL-Policy=<allow|deny>

This is of particular utility when fetching matching intentions, as the
fallthrough for a request that doesn't match any intentions is to
enforce using the default acl policy.
2020-11-12 10:38:32 -06:00
hashicorp-ci c6dd87c79e auto-updated agent/uiserver/bindata_assetfs.go from commit 7243f1f4f 2020-11-12 15:45:53 +00:00
Daniel Nephin a7fec642fc ci: go-test-race switch to exclude list
Most packages should pass the race detector. An exclude list ensures
that new packages are automatically tested with -race.

Also fix a couple small test races to allow more packages to be tested.

Returning readyCh requires a lock because it can be set to nil, and
setting it to nil will race without the lock.

Move the TestServer.Listening calls around so that they properly guard
setting TestServer.l. Otherwise it races.

Remove t.Parallel in a small package. The entire package tests run in a
few seconds, so t.Parallel does very little.

In auto-config, wait for the AutoConfig.run goroutine to stop before
calling readPersistedAutoConfig. Without this change there was a data
race on reading ac.config.
2020-11-11 14:44:57 -05:00
Daniel Nephin 956bff398a grpc: fix grpc metrics
defaultMetrics was being set at package import time, which meant that it received an instance of
the original default. But lib/telemetry.InitTelemetry sets a new global when it is called.

This resulted in the metrics being sent nowhere.

This commit changes defaultMetrics to be a function, so it will return the global instance when
called. Since it is called after InitTelemetry it will return the correct metrics instance.
2020-11-11 14:27:25 -05:00
Matt Keeler 58f98db227
Add a CLI command for retrieving the autopilot configuration. (#9142) 2020-11-11 13:19:02 -05:00
hashicorp-ci aac3729a6c auto-updated agent/uiserver/bindata_assetfs.go from commit 6423a2c10 2020-11-11 17:03:36 +00:00
Matt Keeler 2badb01d30
Add a paramter in state store methods to indicate whether a resource insertion is from a snapshot restoration (#9156)
The Catalog, Config Entry, KV and Session resources potentially re-validate the input as its coming in. We need to prevent snapshot restoration failures due to missing namespaces or namespaces that are being deleted in enterprise.
2020-11-11 11:21:42 -05:00
hashicorp-ci 4c2ca3a516 auto-updated agent/uiserver/bindata_assetfs.go from commit e1d977138 2020-11-11 14:48:38 +00:00
Daniel Nephin e033ad896a
Merge pull request #9149 from joel0/wrap-errors
Use error wrapping to preserve error type info
2020-11-10 18:27:08 -05:00
Daniel Nephin 386eb567f9
Merge pull request #8976 from joel0/wrap-eof
Wrap rpc error object
2020-11-10 17:04:11 -05:00
Joel May 87747ecd34 Use error wrapping to preserve error type info 2020-11-10 21:50:09 +00:00
hashicorp-ci 9fdbc68c92 auto-updated agent/uiserver/bindata_assetfs.go from commit e18d8e299 2020-11-10 16:37:33 +00:00
hashicorp-ci 892fa384fe auto-updated agent/uiserver/bindata_assetfs.go from commit fb6202929 2020-11-10 14:42:02 +00:00
hashicorp-ci 5626983031 auto-updated agent/uiserver/bindata_assetfs.go from commit c8e40ee0d 2020-11-09 17:34:25 +00:00
Matt Keeler 1f40f51a58
Fix a bunch of linter warnings 2020-11-09 09:22:12 -05:00
Matt Keeler 755fb72994
Switch to using the external autopilot module 2020-11-09 09:22:11 -05:00
hashicorp-ci 901df77971 auto-updated agent/uiserver/bindata_assetfs.go from commit 5c0ec13fb 2020-11-09 09:31:52 +00:00
hashicorp-ci eaafa5c17d auto-updated agent/uiserver/bindata_assetfs.go from commit d9672bca8 2020-11-09 09:19:52 +00:00
Mike Morris 9ccb340893
chore: upgrade to gopsutil/v3 (#9118)
* deps: update golang.org/x/sys

* deps: update imports to gopsutil/v3

* chore: make update-vendor
2020-11-06 20:48:38 -05:00
Daniel Nephin e4a78c977d stream: document that Payload must be immutable
If they are sent to EventPublisher.Publish.

Also document that PayloadEvents is expected to come from a subscription and that it is
not immutable.
2020-11-06 13:00:33 -05:00
R.B. Boyer 9b37ea7dcb
Revert "Add namespace support for metrics (OSS) (#9117)" (#9124)
This reverts commit 06b3b017d326853dbb53bc0ec08ce371265c5ce9.
2020-11-06 10:24:32 -06:00
hashicorp-ci 4db32dd6c5 auto-updated agent/uiserver/bindata_assetfs.go from commit 3a68686cc 2020-11-06 15:04:29 +00:00
hashicorp-ci 6e87590a9a auto-updated agent/uiserver/bindata_assetfs.go from commit 848f72f66 2020-11-06 09:31:18 +00:00
Freddy 874efe705f
Add namespace support for metrics (OSS) (#9117) 2020-11-05 18:24:29 -07:00
Daniel Nephin 4fc073b1f4 stream: rename FilterByKey 2020-11-05 19:21:16 -05:00
Daniel Nephin d4cd2fa6a8 stream: Add HasReadPermission to Payload
Required now that filter is a method on PayloadEvents instead of Event
2020-11-05 19:17:18 -05:00
Daniel Nephin 8a26bca020 stream: move event filtering to PayloadEvents
Removes the weirdness around PayloadEvents.FilterByKey
2020-11-05 17:50:17 -05:00
Daniel Nephin dcacfd3548 stream: Remove unused method 2020-11-05 16:49:59 -05:00
R.B. Boyer be9122a8c4
agent: sanitize ui metrics proxy header values on agent/self endpoint (#9104) 2020-11-05 13:25:27 -06:00
Daniel Nephin 621f1db766
Merge pull request #9073 from hashicorp/dnephin/backport-streaming-namespaces
streaming: backport namespace changes
2020-11-05 14:19:10 -05:00
Daniel Nephin cd220e5d6c
Merge pull request #9061 from hashicorp/dnephin/event-fields
stream: support filtering by namespace
2020-11-05 14:18:35 -05:00
hashicorp-ci 70fb9d94d2 auto-updated agent/uiserver/bindata_assetfs.go from commit 6ff094976 2020-11-05 19:12:03 +00:00
hashicorp-ci 17d1fa5f42 auto-updated agent/uiserver/bindata_assetfs.go from commit 1ef18c4b6 2020-11-05 16:10:14 +00:00
R.B. Boyer c2d167d06e
agent: protect the ui metrics proxy endpoint behind ACLs (#9099)
This ensures the metrics proxy endpoint is ACL protected behind a
wildcard `service:read` and `node:read` set of rules. For Consul
Enterprise these will need to span all namespaces:

```
service_prefix "" { policy = "read" }
node_prefix ""    { policy = "read" }

namespace_prefix "" {
  service_prefix "" { policy = "read" }
  node_prefix ""    { policy = "read" }
}
```

This PR contains just the backend changes. The frontend changes to
actually pass the consul token header to the proxy through the JS plugin
will come in another PR.
2020-11-04 12:50:03 -06:00
hashicorp-ci 6645dfcbb1 auto-updated agent/uiserver/bindata_assetfs.go from commit 0f6c0a5c1 2020-11-04 09:37:51 +00:00
hashicorp-ci ebe0ffce30 auto-updated agent/uiserver/bindata_assetfs.go from commit d5d4155e1 2020-11-03 14:14:58 +00:00
hashicorp-ci 0d8a30d8e3 auto-updated agent/uiserver/bindata_assetfs.go from commit 56c2ff56e 2020-11-02 18:43:31 +00:00
hashicorp-ci 0fa484ae8f auto-updated agent/uiserver/bindata_assetfs.go from commit bf32a1799 2020-11-02 16:11:45 +00:00
hashicorp-ci 052303f0fa auto-updated agent/uiserver/bindata_assetfs.go from commit 314eeda95 2020-11-02 14:40:27 +00:00
R.B. Boyer c8c87ec317
agent: introduce path allow list for requests going through the metrics proxy (#9059)
Added a new option `ui_config.metrics_proxy.path_allowlist`. This defaults to `["/api/v1/query", "/api/v1/query_range"]` when the metrics provider is set to `prometheus`.

Requests that do not use one of the allow-listed paths (via exact match) get a 403 Forbidden response instead.
2020-10-30 16:49:54 -05:00
Daniel Nephin 8a017c4f43 structs: add a namespace test for CheckServiceNode.CanRead 2020-10-30 15:07:04 -04:00
Daniel Nephin 78260952b0 cache-type: use namespace in tests
to verify that the namespace is passed through correctly to the server.
2020-10-30 15:07:04 -04:00
Daniel Nephin f6b629852f state: test EventPayloadCheckServiceNode.FilterByKey
Also fix a bug in that function when only one of key or namespace were the empty string.
2020-10-30 14:35:57 -04:00
Daniel Nephin 60df44df4f stream: Add tests for filterByKey with namespace
And fix a bug where a request with a Namespace but no Key would not be properly filtered
2020-10-30 14:35:42 -04:00
Daniel Nephin 318dfbe6e4 stream: Move FilterByKey events to a table
In preparation for adding new tests.
2020-10-30 14:35:28 -04:00
Daniel Nephin 2d0030da39 state: use enterprise meta for creating events 2020-10-30 14:34:04 -04:00
Daniel Nephin b57c7afcbb stream: include the namespace in the snap cache key
Otherwise the wrong snapshot could be returned when the same key is used in different namespaces
2020-10-30 14:34:04 -04:00
Daniel Nephin 8da30fcb9a subscribe: set the request namespace 2020-10-30 14:34:04 -04:00
hashicorp-ci e69d2c99cf auto-updated agent/uiserver/bindata_assetfs.go from commit cf2cfbaf2 2020-10-30 15:27:01 +00:00
R.B. Boyer 67a0d0c426
state: ensure we unblock intentions queries upon the upgrade to config entries (#9062)
1. do a state store query to list intentions as the agent would do over in `agent/proxycfg` backing `agent/xds`
2. upgrade the database and do a fresh `service-intentions` config entry write
3. the blocking query inside of the agent cache in (1) doesn't notice (2)
2020-10-29 15:28:31 -05:00
R.B. Boyer 78014653b3 restore prior signature of test helper so enterprise compiles 2020-10-29 13:52:15 -05:00
hashicorp-ci e970f9699f auto-updated agent/uiserver/bindata_assetfs.go from commit 1d6961248 2020-10-29 18:33:41 +00:00
Michael Montgomery 1c0a46849a Resolves #6074. Adds new option to configure HTTP Server's MaxHeaderBytes with option `-http-max-header-bytes`
Adds tests for behavior
2020-10-29 12:38:19 -05:00
Daniel Nephin 61ce0964a4 stream: remove Event.Key
Makes Payload a type with FilterByKey so that Payloads can implement
filtering by key. With this approach we don't need to expose a Namespace
field on Event, and we don't need to invest micro formats or require a
bunch of code to be aware of exactly how the key field is encoded.
2020-10-28 16:48:04 -04:00
Daniel Nephin 8ef4c0fcc5 state: use go-cmp for comparison
The output of the previous assertions made it impossible to debug the tests without code changes.

With go-cmp comparing the entire slice we can see the full diffs making it easier to debug failures.
2020-10-28 16:33:00 -04:00
Daniel Nephin c106d94742 proto: remove Event.Key field
The field is never used, and the value is available from the payload.
2020-10-28 16:33:00 -04:00
Daniel Nephin ab43236f86 proto: remove Event.Namespace field
All events are part of a single Topic, so we don't need this field.
2020-10-28 16:33:00 -04:00
Daniel Nephin 44da869ed4 stream: Use a no-op event publisher if streaming is disabled 2020-10-28 13:54:19 -04:00
Daniel Nephin eea87e1acf store: use a ReadDB for snapshots
to remove the cyclic dependency between the snapshot handlers and the state.Store
2020-10-28 13:07:42 -04:00
Daniel Nephin cfe0ffde15
Merge pull request #9026 from hashicorp/dnephin/streaming-without-cache-query-param
streaming: rename config and remove requirement for cache=1
2020-10-28 12:33:25 -04:00
Daniel Nephin 03d2be03e7
Merge pull request #8618 from hashicorp/dnephin/remove-txn-readtxn
state: Use ReadTxn everywhere
2020-10-28 12:32:47 -04:00
Kyle Havlovitz ecf0f0b2aa
Merge pull request #9053 from hashicorp/vault-token-lookupself
connect: Use the lookup-self endpoint for Vault token
2020-10-27 14:34:03 -07:00
Daniel Nephin 40cb72fe06 agent/grpc: add connection count metrics
Gauge metrics are great for understanding the current state, but can somtimes hide problems
if there are many disconnect/reconnects.

This commit adds counter metrics for connections and streams to make it easier to see the
count of newly created connections and streams.
2020-10-27 16:49:49 -04:00
Daniel Nephin 64284ed91a agent/grpc: rename metrics
These new names should make it easier to add counter metics with similar prefixes
2020-10-27 16:49:49 -04:00
Daniel Nephin 72430b9125
Merge pull request #8961 from hashicorp/dnephin/grpc-resolve-node-id
agent/grpc: fix some test flakes and handle duplicate server IDs in the pool
2020-10-27 16:47:37 -04:00
Daniel Nephin a0e017791f
Merge pull request #8998 from hashicorp/dnephin/lib-ttlcache
lib/ttlcache: extract a new package from agent/cache
2020-10-27 16:43:10 -04:00
Daniel Nephin 87a43b0ba7
Merge pull request #8987 from hashicorp/dnephin/stream-filter
streaming: apply filter to a single item
2020-10-27 16:39:43 -04:00
Daniel Nephin 6e34759442 agent/grpc: Add an integration test for ClientPool with TLS
Also deregister the resolver.Builder in tests.
2020-10-27 16:34:18 -04:00
Daniel Nephin 87793cd090 agent/grpc: pass metrics to constructor
Instead of referencing a package var. This does not fix the flaky test, but it seems more correct.
2020-10-27 16:34:17 -04:00
Daniel Nephin 70fea7a77e agent/grpc: fix a flaky test by performing more retries
Instead of using retry.Run, which appears to have problems in some cases where it does not
emit an error message, use a for loop.

Increase the number of attempts and remove any sleep, since this operation is not that expensive to do
in a tight loop
2020-10-27 16:34:17 -04:00
Daniel Nephin 9b89fb492d agent/grpc: remove misleading warnings from test output
Handle shutdown properly in tests so that the tests don't warn about using a closed connection.
2020-10-27 16:34:16 -04:00
Daniel Nephin 64105079d9 agent/grpc: fix a flake in TestHandler_EmitsStats 2020-10-27 16:34:16 -04:00
Daniel Nephin 7e338693a8 agent/grpc: use a separate channel for closing the Accept
Closing l.conns can lead to a race and a 'panic: send on closed chan' when a
connection is in the middle of being handled when the server is shutting down.

Found using '-race -count=800'
2020-10-27 16:34:15 -04:00
Daniel Nephin e640d47319 agent/grpc/resolver: namespace the server ID with the DC name
So that if two datacenters end up with overlapping serverIDs we don't send requests to the wrong server
2020-10-27 16:34:15 -04:00
Kyle Havlovitz 1c0608eeeb connect: Use the lookup-self endpoint for Vault token 2020-10-27 13:03:45 -07:00
hashicorp-ci e11b76ce80 auto-updated agent/uiserver/bindata_assetfs.go from commit f4208b5fb 2020-10-27 14:56:48 +00:00
hashicorp-ci e5b54d9be3 auto-updated agent/uiserver/bindata_assetfs.go from commit 30da884d5 2020-10-27 14:31:16 +00:00
hashicorp-ci 707ef19716 auto-updated agent/uiserver/bindata_assetfs.go from commit ed6a2c150 2020-10-27 14:09:17 +00:00
hashicorp-ci 338002054f auto-updated agent/uiserver/bindata_assetfs.go from commit 827e53694 2020-10-27 13:36:44 +00:00
hashicorp-ci 5ae5a4b800 auto-updated agent/uiserver/bindata_assetfs.go from commit 52d7283cd 2020-10-26 19:56:11 +00:00
Paul Banks df1eec292f
UI metrics provider dc (#9001)
* Plumb Datacenter and Namespace to metrics provider in preparation for them being usable.

* Move metrics loader/status to a new component and show reason for being disabled.

* Remove stray console.log

* Rebuild AssetFS to resolve conflicts

* Yarn upgrade

* mend
2020-10-26 19:48:23 +00:00
hashicorp-ci fea7f184ea auto-updated agent/uiserver/bindata_assetfs.go from commit c4f027fa0 2020-10-26 16:55:52 +00:00
Daniel Nephin 68a0fd3f8c streaming: disable streaming when requesting connect events
Until the correct events are created for terminating gateways.
2020-10-26 11:55:49 -04:00
Daniel Nephin abd8cfcfe9 state: disable streaming connect topic 2020-10-26 11:49:47 -04:00
hashicorp-ci 03212abfca auto-updated agent/uiserver/bindata_assetfs.go from commit 948917c6b 2020-10-26 09:34:58 +00:00
Daniel Nephin 1dcbfd17c3 health: change the name of UseStreamingBackend config
Remove it from the cache section, and update the docs.
2020-10-23 17:47:01 -04:00
Daniel Nephin 6650146a74 health: use streaming, even when cache=1 is not set 2020-10-23 17:39:55 -04:00
R.B. Boyer 0a80e82f21
server: config entry replication now correctly uses namespaces in comparisons (#9024)
Previously config entries sharing a kind & name but in different
namespaces could occasionally cause "stuck states" in replication
because the namespace fields were ignored during the differential
comparison phase.

Example:

Two config entries written to the primary:

    kind=A,name=web,namespace=bar
    kind=A,name=web,namespace=foo

Under the covers these both get saved to memdb, so they are sorted by
all 3 components (kind,name,namespace) during natural iteration. This
means that before the replication code does it's own incomplete sort,
the underlying data IS sorted by namespace ascending (bar comes before
foo).

After one pass of replication the primary and secondary datacenters have
the same set of config entries present. If
"kind=A,name=web,namespace=bar" were to be deleted, then things get
weird. Before replication the two sides look like:

primary: [
    kind=A,name=web,namespace=foo
]
secondary: [
    kind=A,name=web,namespace=bar
    kind=A,name=web,namespace=foo
]

The differential comparison phase walks these two lists in sorted order
and first compares "kind=A,name=web,namespace=foo" vs
"kind=A,name=web,namespace=bar" and falsely determines they are the SAME
and are thus cause an update of "kind=A,name=web,namespace=foo". Then it
compares "<nothing>" with "kind=A,name=web,namespace=foo" and falsely
determines that the latter should be DELETED.

During reconciliation the deletes are processed before updates, and so
for a brief moment in the secondary "kind=A,name=web,namespace=foo" is
erroneously deleted and then immediately restored.

Unfortunately after this replication phase the final state is identical
to the initial state, so when it loops around again (rate limited) it
repeats the same set of operations indefinitely.
2020-10-23 13:41:54 -05:00
Daniel Nephin f9b2834171 state: convert the remaining functions to ReadTxn
Required also converting some of the transaction functions to WriteTxn
because TxnRO() called the same helper as TxnRW.

This change allows us to return a memdb.Txn for read-only txn instead of
wrapping them with state.txn.
2020-10-23 14:29:22 -04:00
Daniel Nephin 26387cdc0e
Merge pull request #8975 from hashicorp/dnephin/stream-close-on-unsub
stream: close the subscription on Unsubscribe
2020-10-23 12:58:12 -04:00
Freddy d23038f94f
Add HasExact to topology endpoint (#9010) 2020-10-23 10:45:41 -06:00
hashicorp-ci 8fa2b814ad auto-updated agent/uiserver/bindata_assetfs.go from commit eedee07e7 2020-10-23 16:32:06 +00:00
hashicorp-ci d1a6880699 auto-updated agent/uiserver/bindata_assetfs.go from commit 7559f64d0 2020-10-23 14:45:43 +00:00
hashicorp-ci e174a0c274 auto-updated agent/uiserver/bindata_assetfs.go from commit 2a8a80638 2020-10-23 13:50:57 +00:00
hashicorp-ci 0b7ee99f23 auto-updated agent/uiserver/bindata_assetfs.go from commit 45554e4e9 2020-10-23 08:32:18 +00:00
R.B. Boyer 2183842f0e
connect: add support for envoy 1.16.0, drop support for 1.12.x, and bump point releases as well (#8944)
Supported versions will be: "1.16.0", "1.15.2", "1.14.5", "1.13.6"
2020-10-22 13:46:19 -05:00
Daniel Nephin 56050a1dd1
Merge pull request #8973 from hashicorp/dnephin/resolver-balance
agent/router: refactor calculation of delay between rebalances.
2020-10-22 13:52:10 -04:00
Daniel Nephin fb8b68a6ec stream: close the subscription on Unsubscribe 2020-10-22 13:39:27 -04:00
Daniel Nephin 1d03a7a8a4
Merge pull request #8924 from ShimmerGlass/fix-sidecar-deregister-after-restart
Fix: service LocallyRegisteredAsSidecar property is not persisted
2020-10-22 13:26:55 -04:00
Daniel Nephin 193c89429d
Merge pull request #9004 from hashicorp/dnephin/dns-small-cleanup-1
dns: remove goto INVALID and a naked return
2020-10-21 18:41:11 -04:00
Daniel Nephin 19cffcb1f3
Merge pull request #8970 from hashicorp/dnephin/resolver-balance-test
agent/router: improve  the test for refreshServerRebalanceTimer
2020-10-21 18:40:32 -04:00
Daniel Nephin 1e1f963064
Merge pull request #8771 from amenzhinsky/fix-grpc-use-tls-mapping
Fix GRPCUseTLS flag HTTP API mapping
2020-10-21 18:37:11 -04:00
Daniel Nephin aa06ae91ce dns: remove goto INVALID and a naked return
This commit is one small step toward modernizing and making the DNSServer readable.
2020-10-21 15:16:03 -04:00
Daniel Nephin 09d62f1df0 lib/ttlcache: unexport key and additional godoc 2020-10-20 19:16:03 -04:00
Daniel Nephin 2601998766 lib/ttlcache: add a constant for NotIndexed 2020-10-20 19:10:20 -04:00
Daniel Nephin 0beaced90f cache: fix a bug with Prepopulate
Prepopulate was setting entry.Expiry.HeapIndex to 0. Previously this would result in a call to heap.Fix(0)
which wasn't correct, but was also not really a problem because at worse it would re-notify.

With the recent change to extract cachettl it was changed to call Update(idx), which would have updated
the wrong entry.

A previous commit removed the setting of entry.Expiry so that the HeapIndex would be reported
as -1, and this commit adds a test and handles the -1 heap index.
2020-10-20 19:10:20 -04:00
Daniel Nephin 9d5b738cdb lib/ttlcache: extract package from agent/cache 2020-10-20 19:10:20 -04:00
Daniel Nephin 909b8e674e cache: export ExpiryHeap
and hide internal methods on an unexported type, so that when it is extrated those methods are not exported.
2020-10-20 19:10:20 -04:00
Daniel Nephin d3742a1d0e cache: Refactor heap.notify to make it more explicit.
And remove duplicate notifications.

Instead of performing the check in the heap implementation, check the
index in the higher level interface (Add,Remove,Update) and notify if one
of the relevant indexes is 0.
2020-10-20 19:10:20 -04:00
Daniel Nephin a96646c562 cache: Move more of the expiryLoop into the Heap 2020-10-20 19:10:20 -04:00
Daniel Nephin b6f24c6554 cache: extract cache eviction heap
Start creating an interface that doesn't require using heap and hides more of the
entry internals.
2020-10-20 19:10:19 -04:00
Daniel Nephin 312a3bb9b1 streaming: apply filter to a single item
Instead of the whole map. This should save a lot of time performing reflecting on a large map.
The filter does not change, so there is no reason to re-apply it to older entries.
2020-10-19 18:24:02 -04:00
hashicorp-ci 5de8c073cc auto-updated agent/uiserver/bindata_assetfs.go from commit 2773daedc 2020-10-19 16:35:48 +00:00
hashicorp-ci 054483d5cf auto-updated agent/uiserver/bindata_assetfs.go from commit b24c2227d 2020-10-19 16:19:40 +00:00
hashicorp-ci f2d511b9e5 auto-updated agent/uiserver/bindata_assetfs.go from commit 02dcd422f 2020-10-19 14:50:52 +00:00
hashicorp-ci ef43c1a09e auto-updated agent/uiserver/bindata_assetfs.go from commit bf8bb7415 2020-10-19 14:25:54 +00:00
Joel May e603d3fd98 Wrap rpc error object 2020-10-16 00:42:05 +00:00
Daniel Nephin a3f922249e agent/router: refactor calculation of delay between rebalances.
This change attempts to make the delay logic more obvious by:

* remove indirection, inline a bunch of function calls
* move all the code and constants next to each other
* replace the two constant values with a single value
* reword the comments.
2020-10-15 15:59:36 -04:00
Daniel Nephin da1e45745c agent/router: Add bounds test cases 2020-10-15 14:43:29 -04:00
Daniel Nephin 9bbfc53685 router: organize the test by number of servers
And adddd some additional cases to show where the minimum value stops being used
2020-10-15 13:53:37 -04:00
Daniel Nephin 2ac2503604 router: make refreshServerRebalanceTimer test a lot more strict 2020-10-15 12:05:07 -04:00
hashicorp-ci d15d2b5b2c auto-updated agent/uiserver/bindata_assetfs.go from commit d08e13e5b 2020-10-15 14:49:01 +00:00
hashicorp-ci e5ace85562 auto-updated agent/uiserver/bindata_assetfs.go from commit 5fd79ba40 2020-10-14 08:07:29 +00:00
Mathilde Gilles a1ec792acc Fix: service LocallyRegisteredAsSidecar property is not persisted
When a service is deregistered, we check whever matching services were
registered as sidecar along with it and deregister them as well.
To determine if a service is indeed a sidecar we check the
structs.ServiceNode.LocallyRegisteredAsSidecar property. However, to
avoid interal API leakage, it is excluded from JSON serialization,
meaning it is not persisted to disk either.
When the agent is restarted, this property lost and sidecars are no
longer deregistered along with their parent service.
To fix this, we now specifically save this property in the persisted
service file.
2020-10-13 19:38:58 +02:00
hashicorp-ci 646e1f369c auto-updated agent/uiserver/bindata_assetfs.go from commit 7afcd7755 2020-10-13 16:43:31 +00:00
Mike Morris 528b0f4621 chore: regenerate bindata_assetfs 2020-10-12 15:44:33 -04:00
Mike Morris f2443e0745 Merge branch 'master' into release/1.9.0-beta1 2020-10-12 15:32:58 -04:00
hashicorp-ci d2f62d64cb
update bindata_assetfs.go 2020-10-12 18:30:30 +00:00
hashicorp-ci 72988bae34 auto-updated agent/uiserver/bindata_assetfs.go from commit 868cd47a6 2020-10-12 16:28:44 +00:00
Pierre Souchay 54f9f247f8
Consul Service meta wrongly computes and exposes non_voter meta (#8731)
* Consul Service meta wrongly computes and exposes non_voter meta

In Serf Tags, entreprise members being non-voters use the tag
`nonvoter=1`, not `non_voter = false`, so non-voters in members
were wrongly displayed as voter.

Demonstration:

```
consul members -detailed|grep voter
consul20-hk5 10.200.100.110:8301   alive   acls=1,build=1.8.4+ent,dc=hk5,expect=3,ft_fs=1,ft_ns=1,id=xxxxxxxx-5629-08f2-3a79-10a1ab3849d5,nonvoter=1,port=8300,raft_vsn=3,role=consul,segment=<all>,use_tls=1,vsn=2,vsn_max=3,vsn_min=2,wan_join_port=8302
```

* Added changelog

* Added changelog entry
2020-10-09 17:18:24 -04:00
hashicorp-ci f58806fbab auto-updated agent/uiserver/bindata_assetfs.go from commit 851705e93 2020-10-09 21:01:02 +00:00
hashicorp-ci 89977d1467 auto-updated agent/uiserver/bindata_assetfs.go from commit 27048a061 2020-10-09 20:35:59 +00:00