luxolus/open-consul
2
0
Fork 0
Code Issues 1 Pull requests Packages Releases Wiki Activity
20098 commits 1 branch 1 tag 358 MiB
Commit graph

160 commits

Author SHA1 Message Date
Chris S. Kim d73a9522cb
Add support for streaming CA roots to peers (#13260) 
Sender watches for changes to CA roots and sends
them through the replication stream. Receiver saves
CA roots to tablePeeringTrustBundle
 2022-05-26 15:24:09 -04:00
R.B. Boyer bc10055edc
peering: replicate expected SNI, SPIFFE, and service protocol to peers (#13218) 
The importing peer will need to know what SNI and SPIFFE name
corresponds to each exported service. Additionally it will need to know
at a high level the protocol in use (L4/L7) to generate the appropriate
connection pool and local metrics.

For replicated connect synthetic entities we edit the `Connect{}` part
of a `NodeService` to have a new section:

    {
      "PeerMeta": {
        "SNI": [
          "web.default.default.owt.external.183150d5-1033-3672-c426-c29205a576b8.consul"
        ],
        "SpiffeID": [
          "spiffe://183150d5-1033-3672-c426-c29205a576b8.consul/ns/default/dc/dc1/svc/web"
        ],
        "Protocol": "tcp"
      }
    }

This data is then replicated and saved as-is at the importing side. Both
SNI and SpiffeID are slices for now until I can be sure we don't need
them for how mesh gateways will ultimately work.
 2022-05-25 12:37:44 -05:00
R.B. Boyer 69191fc0da
peering: disable requirement for mesh gateways initially (#13213) 2022-05-25 10:13:23 -05:00
alex 451dc50f4f
peering: expose IsLeader, hung up on dialer if follower (#13164) 
Signed-off-by: acpana <8968914+acpana@users.noreply.github.com>

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>
 2022-05-23 11:30:58 -07:00
R.B. Boyer 63a9175bd6
peering: accept replication stream of discovery chain information at the importing side (#13151) 2022-05-19 16:37:52 -05:00
R.B. Boyer 91691eca87 peering: replicate discovery chains information to importing peers 
Treat each exported service as a "discovery chain" and replicate one
synthetic CheckServiceNode for each chain and remote mesh gateway.

The health will be a flattened generated check of the checks for that
mesh gateway node.
 2022-05-19 14:21:44 -05:00
R.B. Boyer bf05e8c1f1 prefactor some functions out of the monolithic file 2022-05-19 14:21:29 -05:00
Freddy 6c868b6c0e
Patches to peering initiation for POC demo (#13076) 
Co-authored-by: R.B. Boyer <rb@hashicorp.com>
 2022-05-13 13:01:00 -06:00
Freddy 160acdf876
Actually block when syncing subscriptions (#13066) 
By changing to use WatchCtx we will actually block for changes to the peering list. WatchCh creates a goroutine to collect errors from WatchCtx and returns immediately.

The existing behavior wouldn't result in a tight loop because of the rate limiting in the surrounding function, but it would still lead to more work than is necessary.
 2022-05-12 17:36:14 -06:00
Evan Culver 535e811020
peering: add TrustBundleListByService endpoint (#13048) 2022-05-12 15:58:22 -07:00
Freddy 8894365c5a
[OSS] Add upsert handling for receiving CheckServiceNode (#13061) 2022-05-12 15:04:44 -06:00
R.B. Boyer b932d0dabc
test: ensure this package uses freeport for port allocation (#13036) 2022-05-11 14:20:50 -05:00
R.B. Boyer c855df87ec
remove remaining shim runStep functions (#13015) 
Wraps up the refactor from #13013
 2022-05-10 16:24:45 -05:00
R.B. Boyer 9ad10318cd
add general runstep test helper instead of copying it all over the place (#13013) 2022-05-10 15:25:51 -05:00
FFMMM 76a6647700
expose meta tags for peering (#12964) 2022-05-09 13:47:37 -07:00
R.B. Boyer 809344a6f5
peering: initial sync (#12842) 
- Add endpoints related to peering: read, list, generate token, initiate peering
- Update node/service/check table indexing to account for peers
- Foundational changes for pushing service updates to a peer
- Plumb peer name through Health.ServiceNodes path

see: ENT-1765, ENT-1280, ENT-1283, ENT-1283, ENT-1756, ENT-1739, ENT-1750, ENT-1679,
     ENT-1709, ENT-1704, ENT-1690, ENT-1689, ENT-1702, ENT-1701, ENT-1683, ENT-1663,
     ENT-1650, ENT-1678, ENT-1628, ENT-1658, ENT-1640, ENT-1637, ENT-1597, ENT-1634,
     ENT-1613, ENT-1616, ENT-1617, ENT-1591, ENT-1588, ENT-1596, ENT-1572, ENT-1555

Co-authored-by: R.B. Boyer <rb@hashicorp.com>
Co-authored-by: freddygv <freddy@hashicorp.com>
Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
Co-authored-by: Evan Culver <eculver@hashicorp.com>
Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com>
 2022-04-21 17:34:40 -05:00
FFMMM cf7e6484aa
add more labels to RequestRecorder (#12727) 
Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>
Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>
 2022-04-12 10:50:25 -07:00
FFMMM 0f68bf879a
[rpc/middleware][consul] plumb intercept off, add server level happy test (#12692) 2022-04-06 14:33:05 -07:00
FFMMM 6bdde40d5e
lower log to trace (#12708) 2022-04-06 11:37:08 -07:00
FFMMM 8b184197b3
polish rpc.service.call metric behavior (#12624) 2022-03-31 10:49:37 -07:00
FFMMM 560f8cbc89
fix bad oss sync, use gauges not counters (#12611) 2022-03-24 14:41:30 -07:00
FFMMM 76d8798590
factor out recording func, add unit tests (#12585) 
Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>
 2022-03-22 09:31:54 -07:00
Dan Upton fb441e323a
Restructure gRPC server setup (#12586) 
OSS sync of enterprise changes at 0b44395e
 2022-03-22 12:40:24 +00:00
FFMMM 08f2838b78
pre register new rpc metric, rename metric (#12582) 2022-03-21 17:26:32 -07:00
FFMMM 3c08843847
[sync oss] add net/rpc interceptor implementation (#12573) 
* sync ent changes from 866dcb0667

Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>

* update oss go.mod

Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>
 2022-03-17 16:02:26 -07:00
Dan Upton ebdda4848f
streaming: split event buffer by key (#12080) 2022-01-28 12:27:00 +00:00
Giulio Micheloni 10cdc0a5c8
Merge branch 'main' into serve-panic-recovery 2021-11-06 16:12:06 +01:00
Daniel Nephin db29ad346b acl: remove id and revision from Policy constructors 
The fields were removed in a previous commit.

Also remove an unused constructor for PolicyMerger
 2021-11-05 15:45:08 -04:00
Daniel Nephin 88c6aeea34 acl: remove legacy arg to store.ACLTokenSet 
And remove the tests for legacy=true
 2021-10-25 17:25:14 -04:00
Giulio Micheloni 10814d934e Merge branch 'main' of https://github.com/hashicorp/consul into hashicorp-main 2021-10-16 16:59:32 +01:00
R.B. Boyer ba13416b57
grpc: strip local ACL tokens from RPCs during forwarding if crossing datacenters (#11099) 
Fixes #11086
 2021-09-22 13:14:26 -05:00
Giulio Micheloni 10b03c3f4e
Merge branch 'main' into serve-panic-recovery 2021-08-22 20:31:11 +02:00
Giulio Micheloni 465e9fecda grpc, xds: recovery middleware to return and log error in case of panic 
1) xds and grpc servers:
   1.1) to use recovery middleware with callback that prints stack trace to log
   1.2) callback turn the panic into a core.Internal error
2) added unit test for grpc server
 2021-08-22 19:06:26 +01:00
R.B. Boyer 61f1c01b83
agent: ensure that most agent behavior correctly respects partition configuration (#10880) 2021-08-19 15:09:42 -05:00
R.B. Boyer e50e13d2ab
state: partition nodes and coordinates in the state store (#10859) 
Additionally:

- partitioned the catalog indexes appropriately for partitioning
- removed a stray reference to a non-existent index named "node.checks"
 2021-08-17 13:29:39 -05:00
Daniel Nephin b6d9d0d9f7 acl: remove many instances of authz == nil 2021-07-30 13:58:35 -04:00
R.B. Boyer 254557a1f6
sync changes to oss files made in enterprise (#10670) 2021-07-22 13:58:08 -05:00
R.B. Boyer 62ac98b564
agent/structs: add a bunch more EnterpriseMeta helper functions to help with partitioning (#10669) 2021-07-22 13:20:45 -05:00
Daniel Nephin 94820e67a8 structs: remove EnterpriseMeta.GetNamespace 
I added this recently without realizing that the method already existed and was named
NamespaceOrEmpty. Replace all calls to GetNamespace with NamespaceOrEmpty or NamespaceOrDefault
as appropriate.
 2021-03-09 15:17:26 -05:00
Daniel Nephin 88bbde56da agent: add a test for streaming in the service health endpoint 
Co-authored-by: Paul Banks <banks@banksco.de>
 2021-02-25 14:08:10 -05:00
Daniel Nephin c40d063a0e structs: rename EnterpriseMeta constructor 
To match the Go convention.
 2021-02-16 14:45:43 -05:00
Daniel Nephin ef0999547a testing: skip slow tests with -short 
Add a skip condition to all tests slower than 100ms.

This change was made using `gotestsum tool slowest` with data from the
last 3 CI runs of master.
See https://github.com/gotestyourself/gotestsum#finding-and-skipping-slow-tests

With this change:

```
$ time go test -count=1 -short ./agent
ok      github.com/hashicorp/consul/agent       0.743s

real    0m4.791s

$ time go test -count=1 -short ./agent/consul
ok      github.com/hashicorp/consul/agent/consul        4.229s

real    0m8.769s
```
 2020-12-07 13:42:55 -05:00
Daniel Nephin e4a78c977d stream: document that Payload must be immutable 
If they are sent to EventPublisher.Publish.

Also document that PayloadEvents is expected to come from a subscription and that it is
not immutable.
 2020-11-06 13:00:33 -05:00
Daniel Nephin d4cd2fa6a8 stream: Add HasReadPermission to Payload 
Required now that filter is a method on PayloadEvents instead of Event
 2020-11-05 19:17:18 -05:00
Daniel Nephin 621f1db766
Merge pull request #9073 from hashicorp/dnephin/backport-streaming-namespaces 
streaming: backport namespace changes
 2020-11-05 14:19:10 -05:00
Daniel Nephin cd220e5d6c
Merge pull request #9061 from hashicorp/dnephin/event-fields 
stream: support filtering by namespace
 2020-11-05 14:18:35 -05:00
Daniel Nephin 8a017c4f43 structs: add a namespace test for CheckServiceNode.CanRead 2020-10-30 15:07:04 -04:00
Daniel Nephin 8da30fcb9a subscribe: set the request namespace 2020-10-30 14:34:04 -04:00
Daniel Nephin 61ce0964a4 stream: remove Event.Key 
Makes Payload a type with FilterByKey so that Payloads can implement
filtering by key. With this approach we don't need to expose a Namespace
field on Event, and we don't need to invest micro formats or require a
bunch of code to be aware of exactly how the key field is encoded.
 2020-10-28 16:48:04 -04:00
Daniel Nephin c106d94742 proto: remove Event.Key field 
The field is never used, and the value is available from the payload.
 2020-10-28 16:33:00 -04:00
Daniel Nephin ab43236f86 proto: remove Event.Namespace field 
All events are part of a single Topic, so we don't need this field.
 2020-10-28 16:33:00 -04:00
Daniel Nephin 44da869ed4 stream: Use a no-op event publisher if streaming is disabled 2020-10-28 13:54:19 -04:00
Daniel Nephin fb8b68a6ec stream: close the subscription on Unsubscribe 2020-10-22 13:39:27 -04:00
Daniel Nephin f06fd96d3a subscribe: add test cases for newEventFromStreamEvent 2020-10-08 18:48:17 -04:00
Daniel Nephin ea95908f63 subscribe: Add steps to rpc/subscribe tests 
To make them easier to follow
 2020-10-08 15:38:01 -04:00
Daniel Nephin e0236b5a9f
Merge pull request #8818 from hashicorp/streaming/add-subscribe-service-batch-events 
stream: handle batch events as a special case of Event
 2020-10-07 21:25:32 -04:00
Daniel Nephin eb6f2a8d72 structs: add CheckServiceNode.CanRead 
And use it from the subscribe endpoint.
 2020-10-07 18:15:13 -04:00
Daniel Nephin ad29cf4f94 stream: Return a single event from a subscription.Next 
Handle batch events as a single event
 2020-10-06 13:18:20 -04:00
Daniel Nephin 3183b9ebb3 subscribe: update to use NewSnapshotToFollow event 2020-10-06 12:49:35 -04:00
Daniel Nephin fa115c6249 Move agent/subscribe -> agent/rpc/subscribe 2020-10-06 12:49:35 -04:00