Commit graph

2468 commits

Author SHA1 Message Date
Daniel Nephin bda19cb71e submatview: remove notifyUpdateLocked from reset
This call appears to only be necessary because reset() was called from
NewMaterializer.

This commit has the constructor set a default value for updateCh, and
removes both the call to reset() from New(), and the call to
notifyUpdateLocked() from reset().

This should ensure that we do not notify the Fetch() call before we have new
values to report.
2020-10-06 13:22:02 -04:00
Daniel Nephin e5813bd9d6 submatview: remove method receiver from handlers 2020-10-06 13:22:02 -04:00
Daniel Nephin f857aef4a8 submatview: add a test for handling of NewSnapshotToFollow
Also add some godoc
Rename some vars and functions
Fix a data race in the new cache test for entry closing.
2020-10-06 13:22:02 -04:00
Daniel Nephin 58cf09247b submatview: refactor Materializer
Refactor of Materializer.Run
Use handlers to manage state in Materializer
Rename Materializer receiver
rename m.l to m.lock, and flip some conditionals to remove the negative.
Improve godoc, rename Deps, move resetErr, and pass err into notifyUpdate
Update for NewSnapshotToFollow events
Refactor to move context cancel out of Materializer
2020-10-06 13:22:02 -04:00
Daniel Nephin e8c7881196 submatview: Move the 'use materialize from result.State' logic
No need to do all this other work if we have one already.

This logic moved closer to this call site 3 times during the process
of refactoring.
2020-10-06 13:22:02 -04:00
Daniel Nephin 3bb252888b submatview: Move Materializer to submatview package 2020-10-06 13:22:02 -04:00
Daniel Nephin d24e243f70 submatview: Refactor MaterializeView
Replace InitFilter with Reset.
Removes the need to store a fatalErr and the cache-type, and removes the need to recreate the filter
each time.
Pass dependencies into MaterializedView.
Remove context from MaterializedView.
Rename state to view.
Rename MaterialziedView to Materialzier.
Rename to NewMaterializer
Pass in retry.Waiter
2020-10-06 13:22:02 -04:00
Daniel Nephin 50846a96ff cache-types: Update Streaming health cache-type
To use latest protobuf types
2020-10-06 13:22:02 -04:00
Daniel Nephin e5d37bdf23 agent/cache: Add cache-type and materialized view for streaming health
Extracted from d97412ce4c399a35b41bbdae2716f0e32dce80bf

Co-authored-by: Paul Banks <banks@banksco.de>
2020-10-06 13:21:57 -04:00
Daniel Nephin 3183b9ebb3 subscribe: update to use NewSnapshotToFollow event 2020-10-06 12:49:35 -04:00
Daniel Nephin fa115c6249 Move agent/subscribe -> agent/rpc/subscribe 2020-10-06 12:49:35 -04:00
Daniel Nephin 011109a6f6 subscirbe: extract streamID and logging from Subscribe
By extracting all of the tracing logic the core logic of the Subscribe
endpoint is much easier to read.
2020-10-06 12:49:35 -04:00
Daniel Nephin 4c4441997a subscribe: add integration test for acl token updates 2020-10-06 12:49:35 -04:00
Daniel Nephin 3deccdde56 subscribe: add integration test for filtering events by acl 2020-10-06 12:49:35 -04:00
Daniel Nephin cb6ce4ab0c subscribe: Add an integration test for forward to DC 2020-10-06 12:49:35 -04:00
Daniel Nephin f5f4a3d803 subscribe: add first integration test for Server 2020-10-06 12:49:35 -04:00
Daniel Nephin 371ec2d70a subscribe: add a stateless subscribe service for the gRPC server
With a Backend that provides access to the necessary dependencies.
2020-10-06 12:49:35 -04:00
Daniel Nephin 2f6c98ee0e subscribe: add commented out test cases
Co-authored-by: Paul Banks <banks@banksco.de>
2020-10-06 12:49:35 -04:00
Daniel Nephin f73432be66 subscribe: add initial impl from streaming-rpc-final branch
Co-authored-by: Paul Banks <banks@banksco.de>
2020-10-06 12:49:35 -04:00
Daniel Nephin ae433947a4
Merge pull request #8799 from hashicorp/streaming/rename-framing-events
stream: remove EndOfEmptySnapshot, add NewSnapshotToFollow
2020-10-06 12:42:58 -04:00
Daniel Nephin d5ac6e8f09
Merge pull request #8806 from hashicorp/dnephin/service-health-interface
rpcclient: Add health.Client and use it in http and dns
2020-10-06 12:13:09 -04:00
R.B. Boyer a77b518542
server: create new memdb table for storing system metadata (#8703)
This adds a new very tiny memdb table and corresponding raft operation
for updating a very small effective map[string]string collection of
"system metadata". This can persistently record a fact about the Consul
state machine itself.

The first use of this feature will come in a later PR.
2020-10-06 10:08:37 -05:00
hashicorp-ci c945bbc6f5 auto-updated agent/uiserver/bindata_assetfs.go from commit a7038b48c 2020-10-06 13:43:27 +00:00
hashicorp-ci 6c78cc6eea auto-updated agent/uiserver/bindata_assetfs.go from commit 5a39be47d 2020-10-06 13:32:27 +00:00
hashicorp-ci 7d4d5cb521 auto-updated agent/uiserver/bindata_assetfs.go from commit 76d3909f3 2020-10-06 08:42:39 +00:00
hashicorp-ci 6d50d8a859 auto-updated agent/uiserver/bindata_assetfs.go from commit e4a0dcf10 2020-10-06 08:36:18 +00:00
Freddy f3d991fb8f
Add default meta to test assertion (#8824) 2020-10-05 15:00:25 -06:00
Daniel Nephin 2706cf9b2a
Merge pull request #8802 from hashicorp/dnephin/extract-lib-retry
lib/retry - extract a new package from lib/retry.go
2020-10-05 14:22:37 -04:00
hashicorp-ci 9eb769d242 auto-updated agent/uiserver/bindata_assetfs.go from commit f26201a7a 2020-10-05 17:12:56 +00:00
freddygv 82a17ccee6 Do not evaluate discovery chain for topology upstreams 2020-10-05 10:24:50 -06:00
freddygv 63c50e15bc Single DB txn for ServiceTopology and other PR comments 2020-10-05 10:24:50 -06:00
freddygv 263bd9dd92 Add topology HTTP endpoint 2020-10-05 10:24:50 -06:00
freddygv 7c11580e93 Add topology RPC endpoint 2020-10-05 10:24:50 -06:00
freddygv 21c4708fe9 Add topology ACL filter 2020-10-05 10:24:50 -06:00
freddygv ac54bf99b3 Add func to combine up+downstream queries 2020-10-05 10:24:50 -06:00
freddygv 160a6539d1 factor in discovery chain when querying up/downstreams 2020-10-05 10:24:50 -06:00
freddygv 214b25919f support querying upstreams/downstreams from registrations 2020-10-05 10:24:50 -06:00
freddygv 3653045cb0 Add method for downstreams from disco chain 2020-10-05 10:24:50 -06:00
hashicorp-ci 9a546f3170 auto-updated agent/uiserver/bindata_assetfs.go from commit 3a89c7d23 2020-10-05 14:07:13 +00:00
hashicorp-ci c652d2551d auto-updated agent/uiserver/bindata_assetfs.go from commit dfe57406d 2020-10-05 13:53:36 +00:00
hashicorp-ci 63ceb4f15e auto-updated agent/uiserver/bindata_assetfs.go from commit b871837ee 2020-10-05 13:14:40 +00:00
Daniel Nephin 04c5b9adf5 rpcclient: Add health.Client and use it in http and dns
This new package provides a client agent implementation of an interface
for fetching the health of services.

This approach has a number of benefits:

1. It provides a much more explicit interface. Instead of everything
   dependency on `RPC()` and `Cache.Get()` for many unrelated things
   they can depend on a type that are named according to the behaviour
   it provides.

2. It gives us a single place to vary the behaviour and migrate to
   a new form of RPC (gRPC). The current implementation has two options
   (cache, or direct RPC), and in the future we will have more.
   It is also a great opporunity to start adding `context.Context` args
   to these operations, which in the future will allow us to cancel
   the operations.

3. As a concequence of the first, in the Server agent where we make
   these calls we can replace the current in-memory RPC calls with
   a thin adapter for the real method. This removes the `net/rpc`
   machinery from the call in places where it is not needed.

This new package is quite small right now, but I think we can expect it
to grow to a more reasonable size as other RPC calls are replaced.

This change also happens to replace two very similar implementations with
a single implementation.
2020-10-04 18:55:02 -04:00
Daniel Nephin 40aac46cf4 lib/retry: Refactor to reduce the interface surface
Reduce Jitter to one function

Rename NewRetryWaiter

Fix a bug in calculateWait where maxWait was applied before jitter, which would make it
possible to wait longer than maxWait.
2020-10-04 18:12:42 -04:00
Daniel Nephin 0c7f9c72d7 lib/retry: extract a new package from lib 2020-10-04 17:43:01 -04:00
Kit Patella 5524a43f10
Merge pull request #8271 from coignetp/http-metrics-label
Use method and path as labels for http metrics
2020-10-02 13:41:48 -07:00
hashicorp-ci 8e7643ef8e auto-updated agent/uiserver/bindata_assetfs.go from commit 8b409529a 2020-10-02 19:28:38 +00:00
Daniel Nephin 9c5181c897 stream: full test coverage for EventPublisher.Subscribe 2020-10-02 13:46:24 -04:00
Daniel Nephin 0769f54fe1 stream: refactor to support change in framing events
Removing EndOfEmptySnapshot, add NewSnapshotToFollow
2020-10-02 13:41:31 -04:00
Daniel Nephin 5ef630f664
Merge pull request #8769 from hashicorp/streaming/prep-for-subscribe-service
state: use protobuf Topic and and export payload type
2020-10-02 13:30:06 -04:00
Paul Banks 35d249e38f
Merge pull request #8694 from hashicorp/ui-config-metrics
Add config changes for UI metrics
2020-10-01 17:38:03 +01:00