Commit graph

17783 commits

Author SHA1 Message Date
Chris S. Kim 34c0093d44 Add new watch.Map type to refactor proxycfg 2022-07-13 16:05:10 -04:00
Chris S. Kim 0936942b2d Scrub VirtualIPs before exporting 2022-07-13 16:05:10 -04:00
Kyle Havlovitz a7ea6cb771
Merge pull request #13699 from hashicorp/tgate-http2-upstream
Respect http2 protocol for upstreams of terminating gateways
2022-07-13 09:41:15 -07:00
R.B. Boyer c8c6484905
proto: add package prefixes for all proto files where it is safe (#13735)
We cannot do this for "subscribe" and "partition" this easily without
breakage so those are omitted.

Any protobuf message passed around via an Any construct will have the
fully qualified package name embedded in the protobuf as a string. Also
RPC method dispatch will include the package of the service during
serialization.

- We will be passing pbservice and pbpeering through an Any as part of
  peer stream replication.

- We will be exposing two new gRPC services via pbpeering and
  pbpeerstream.
2022-07-13 11:03:27 -05:00
Dan Upton 34140ff3e0
grpc: rename public/private directories to external/internal (#13721)
Previously, public referred to gRPC services that are both exposed on
the dedicated gRPC port and have their definitions in the proto-public
directory (so were considered usable by 3rd parties). Whereas private
referred to services on the multiplexed server port that are only usable
by agents and other servers.

Now, we're splitting these definitions, such that external/internal
refers to the port and public/private refers to whether they can be used
by 3rd parties.

This is necessary because the peering replication API needs to be
exposed on the dedicated port, but is not (yet) suitable for use by 3rd
parties.
2022-07-13 16:33:48 +01:00
R.B. Boyer c880728ab4
peerstream: some cosmetic refactors to make this easier to follow (#13732)
- Use some protobuf construction helper methods for brevity.
- Rename a local variable to avoid later shadowing.
- Rename the Nonce field to be more like xDS's naming.
- Be more explicit about which PeerID fields are empty.
2022-07-13 10:00:35 -05:00
John Cowen dc4302e23f
ui: Remove UNDEFINED state from being undeleteable (#13702)
* ui: Remove UNDEFINED state from being undeleteable

* Fixup node tests
2022-07-13 12:06:16 +01:00
John Cowen 51a8955103
ui: Remove horizontal scrollbar from peering list rows (#13701) 2022-07-13 11:22:49 +01:00
Kyle Havlovitz 0ac7de3bae Use protocol from resolved config entry, not gateway service 2022-07-12 16:23:40 -07:00
Kyle Havlovitz 54d8fe9032 Enable http2 options for grpc protocol 2022-07-12 14:38:44 -07:00
R.B. Boyer 81764a5650
peering: always send the mesh gateway SpiffeID even for tcp services (#13728)
If someone were to switch a peer-exported service from L4 to L7 there
would be a brief SAN validation hiccup as traffic shifted to the mesh
gateway for termination.

This PR sends the mesh gateway SpiffeID down all the time so the clients
always expect a switch.
2022-07-12 11:38:13 -05:00
R.B. Boyer ee5eb5a960
state: prohibit changing an exported tcp discovery chain in a way that would break SAN validation (#13727)
For L4/tcp exported services the mesh gateways will not be terminating
TLS. A caller in one peer will be directly establishing TLS connections
to the ultimate exported service in the other peer.

The caller will be doing SAN validation using the replicated SpiffeID
values shipped from the exporting side. There are a class of discovery
chain edits that could be done on the exporting side that would cause
the introduction of a new SpiffeID value. In between the time of the
config entry update on the exporting side and the importing side getting
updated peer stream data requests to the exported service would fail due
to SAN validation errors.

This is unacceptable so instead prohibit the exporting peer from making
changes that would break peering in this way.
2022-07-12 11:17:33 -05:00
R.B. Boyer 2c329475ce
state: prohibit exported discovery chains to have cross-datacenter or cross-partition references (#13726)
Because peerings are pairwise, between two tuples of (datacenter,
partition) having any exported reference via a discovery chain that
crosses out of the peered datacenter or partition will ultimately not be
able to work for various reasons. The biggest one is that there is no
way in the ultimate destination to configure an intention that can allow
an external SpiffeID to access a service.

This PR ensures that a user simply cannot do this, so they won't run
into weird situations like this.
2022-07-12 11:03:41 -05:00
Michael Klein 4a62ef296d
ui: peer permission handling (#13724)
* Request peering permissions when peerings is active

* Update peering ability to use peering resource

* fix canDelete peer permission to check write permission

* use super call in abilities.peer#canDelete
2022-07-12 16:16:47 +01:00
Chris S. Kim 9f5ab3ec10
Return error if ServerAddresses is empty (#13714) 2022-07-12 11:09:00 -04:00
Michael Klein d25b025468
ui: use environment variable for feature flagging peers (#13703)
* ui: use environment variable for feature flagging peers

* Add documentation for `features`-service

* Allow setting feature flag for peers via bookmarklet

* don't use features service for flagging peers

* add ability for checking if peers feature is enabled

* Use abilities to conditionally use peers feature

* Remove unused features service
2022-07-12 12:02:45 +01:00
Michael Wilkerson affae7ae83
update docs (#13711)
* update docs

* Update website/content/docs/nia/enterprise/index.mdx

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
2022-07-11 15:03:18 -07:00
R.B. Boyer 986f24ce52
proto: ensure buf formatter has been applied to protobufs (#13709) 2022-07-11 13:44:51 -05:00
Jeff Boruszak 671d968ecc
Merge pull request #13693 from hashicorp/docs-cluster-peering-updates
docs: Cluster Peering docs fixes
2022-07-11 12:34:07 -05:00
Nathan Coleman 0bb6078002
Merge pull request #13681 from hashicorp/docs/install-capigw-version-env-var
docs(consul-api-gateway): use VERSION env var in install steps
2022-07-11 10:32:19 -05:00
Nathan Coleman 626704fcda
Update website/content/docs/api-gateway/consul-api-gateway-install.mdx 2022-07-11 11:26:04 -04:00
cskh 2d99304762
feat(cli): enable to delete config entry from an input file (#13677)
* feat(cli): enable to delete config entry from an input file

- A new flag to config delete to delete a config entry in a
  valid config file, e.g., config delete -filename
  intention-allow.hcl
- Updated flag validation; -filename and -kind can't be set
  at the same time
- Move decode config entry method from config_write.go to
  helpers.go for reusing ParseConfigEntry()
- add changelog

Co-authored-by: Dan Upton <daniel@floppy.co>
2022-07-11 10:13:40 -04:00
Kyle Havlovitz 18aacf9b55
Merge pull request #13678 from hashicorp/envoy-prometheus-tls-fix
Fix syntax for envoy bootstrap prometheus secret config
2022-07-08 15:58:19 -07:00
Kyle Havlovitz 3803195a44 Add changelog note 2022-07-08 15:23:00 -07:00
Kyle Havlovitz 616a2da835 Respect http2 protocol for upstreams of terminating gateways 2022-07-08 14:30:45 -07:00
R.B. Boyer 5b801db24b
peering: move peer replication to the external gRPC port (#13698)
Peer replication is intended to be between separate Consul installs and
effectively should be considered "external". This PR moves the peer
stream replication bidirectional RPC endpoint to the external gRPC
server and ensures that things continue to function.
2022-07-08 12:01:13 -05:00
Mike Morris fc8fbda641
Update website/content/docs/api-gateway/consul-api-gateway-install.mdx
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
2022-07-07 17:38:30 -04:00
Mike Morris 28d2ee5ada
Update website/content/docs/api-gateway/consul-api-gateway-install.mdx
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
2022-07-07 17:37:12 -04:00
boruszak 7384eefff0 Clarification around "peering_token.json" and adding Partition names 2022-07-07 16:10:21 -05:00
Chris Thain 27c239b596
Docs: Fix path to consul-ecs Terraform modules (#13689) 2022-07-07 13:30:19 -07:00
R.B. Boyer 40c5c7eee2
server: broadcast the public grpc port using lan serf and update the consul service in the catalog with the same data (#13687)
Currently servers exchange information about their WAN serf port
and RPC port with serf tags, so that they all learn of each other's
addressing information. We intend to make larger use of the new
public-facing gRPC port exposed on all of the servers, so this PR
addresses that by passing around the gRPC port via serf tags and
then ensuring the generated consul service in the catalog has
metadata about that new port as well for ease of non-serf-based lookup.
2022-07-07 13:55:41 -05:00
John Cowen 8c0da8fdfb
ui: Peer Deletion (#13665)
* ui: Peer Deletion (#13665)
* ui: Add sorting peer listing by State (#13684)
* ui: Add filtering peer listing by State (#13685)
2022-07-07 18:23:26 +01:00
John Cowen 8d275ac186
ui: CopyableCode component (#13686)
* ui: CopyableCode component plus switch into existing implementations
2022-07-07 17:42:47 +01:00
boruszak 368d88f9b3 "<service-name" fix - added brackets 2022-07-07 10:08:53 -05:00
Mike Morris 4372e4d3c4 docs(consul-api-gateway): use VERSION env var in install steps 2022-07-06 17:22:05 -04:00
Usha Kodali 922004d46b
Consul on ECS compatibility matrix docs update (#13060) 2022-07-06 12:34:14 -07:00
Kyle Havlovitz c31554ec64 Fix syntax for bootstrap sds secret config 2022-07-06 09:53:40 -07:00
Freddy ed9808c4f1
Parse peer name for virtual IP DNS queries (#13602)
This commit updates the DNS query locality parsing so that the virtual
IP for an imported service can be queried.

Note that:
- Support for parsing a peer in other service discovery queries was not
  added.
- Querying another datacenter for a virtual IP is not supported. This
  was technically allowed in 1.11 but is being rolled back for 1.13
  because it is not a use-case we intended to support. Virtual IPs in
  different datacenters are going to collide because they are allocated
  sequentially.
2022-07-06 10:30:04 -06:00
R.B. Boyer 4ce9651421
test: update mockery use to put mocks into test files (#13656)
--testonly doesn't do anything anymore so switch to --filename instead
2022-07-05 16:57:15 -05:00
Jared Kirschner a5cb3e67d8
Merge pull request #13654 from hashicorp/docs/correct-1.10.x-upgrade-path
docs: improve large version change upgrade path
2022-07-05 14:33:28 -04:00
John Cowen 756072898d
ui: Slight update to peering mocks to more properly match actual (#13664) 2022-07-04 18:49:41 +01:00
John Cowen 9377ac7635
ui: Fixup peering imported/exported service counts (#13662)
* ui: Fix up peer states and counts in the listing
2022-07-04 18:49:21 +01:00
Chris S. Kim 0910c41d95
Revise possible states for a peering. (#13661)
These changes are primarily for Consul's UI, where we want to be more
specific about the state a peering is in.

- The "initial" state was renamed to pending, and no longer applies to
  peerings being established from a peering token.

- Upon request to establish a peering from a peering token, peerings
  will be set as "establishing". This will help distinguish between the
  two roles: the cluster that generates the peering token and the
  cluster that establishes the peering.

- When marked for deletion, peering state will be set to "deleting".
  This way the UI determines the deletion via the state rather than the
  "DeletedAt" field.

Co-authored-by: freddygv <freddy@hashicorp.com>
2022-07-04 10:47:58 -04:00
John Cowen dae1f9d0b8
ui: Add peer searching and sorting (#13634)
* ui: Add peer searching and sorting

Initial name search and sort only, more to come here

* Remove old peerings::search component

* Use @model peers

* ui: Peer listing with dc/ns/partition/name based unique IDs and polling deletion (#13648)

* ui: Add peer repo with listing datasource

* ui: Use data-loader component to use the data-source

* ui: Remove ember-data REST things and Route.model hook

* 10 second not 1 second poll

* Fill out Datacenter and Partition

* route > routeName

* Faker randomised mocks for peering endpoint

* ui: Adds initial peer detail page plus address tab (#13651)
2022-07-04 11:31:58 +01:00
John Cowen c86aedfdd5
ui: Gradual deprecation of old StateChart interface (#13604) 2022-07-04 11:22:14 +01:00
Daniel Upton 687c447701 Changelog entry 2022-07-04 10:48:36 +01:00
Daniel Upton e1d0aff462 proxycfg: server-local intention upstreams data source
This is the OSS portion of enterprise PR 2157.

It builds on the local blocking query work in #13438 to implement the
proxycfg.IntentionUpstreams interface using server-local data.

Also moves the ACL filtering logic from agent/consul into the acl/filter
package so that it can be reused here.
2022-07-04 10:48:36 +01:00
Daniel Upton 21ea217b1d proxycfg: server-local intentions data source
This is the OSS portion of enterprise PR 2141.

This commit provides a server-local implementation of the `proxycfg.Intentions`
interface that sources data from streaming events.

It adds events for the `service-intentions` config entry type, and then consumes
event streams (via materialized views) for the service's explicit intentions and
any applicable wildcard intentions, merging them into a single list of intentions.

An alternative approach I considered was to consume _all_ intention events (via
`SubjectWildcard`) and filter out the irrelevant ones. This would admittedly
remove some complexity in the `agent/proxycfg-glue` package but at the expense
of considerable overhead from waking potentially many thousands of connect
proxies every time any intention is updated.
2022-07-04 10:48:36 +01:00
Daniel Upton 497df1ca3b proxycfg: server-local config entry data sources
This is the OSS portion of enterprise PR 2056.

This commit provides server-local implementations of the proxycfg.ConfigEntry
and proxycfg.ConfigEntryList interfaces, that source data from streaming events.

It makes use of the LocalMaterializer type introduced for peering replication,
adding the necessary support for authorization.

It also adds support for "wildcard" subscriptions (within a topic) to the event
publisher, as this is needed to fetch service-resolvers for all services when
configuring mesh gateways.

Currently, events will be emitted for just the ingress-gateway, service-resolver,
and mesh config entry types, as these are the only entries required by proxycfg
— the events will be emitted on topics named IngressGateway, ServiceResolver,
and MeshConfig topics respectively.

Though these events will only be consumed "locally" for now, they can also be
consumed via the gRPC endpoint (confirmed using grpcurl) so using them from
client agents should be a case of swapping the LocalMaterializer for an
RPCMaterializer.
2022-07-04 10:48:36 +01:00
Jared Kirschner 93edefd4a1 docs: improve large version change upgrade path 2022-07-01 05:47:24 -07:00