Commit graph

19449 commits

Author SHA1 Message Date
Thomas Eckert b01dca96af
Implement BindRoutesToGateways (#15950)
* Stub out bind code
* Move into a new package and flesh out binding
* Fill in the actual binding logic
* Bind to all listeners if not specified
* Move bind code up to gateways package
* Fix resource type check
* Add UpsertRoute to listeners
* Add RemoveRoute to listener
* Implement binding as associated functions
* Pass in gateways to BindRouteToGateways
* Add a bunch of tests
* Fix hopping from one listener on a gateway to another
* Remove parents from HTTPRoute
* Apply suggestions from code review
* Fix merge conflict
* Unify binding into a single variadic function 🙌 @nathancoleman
* Remove vestigial error
* Add TODO on protocol check
2023-01-20 15:11:16 -05:00
Luke Kysow ada3530213
Update gossip.mdx (#16030)
Fix description of topic. I think it was copy pasted incorrectly.
2023-01-20 12:10:50 -08:00
cskh 58e3ea5b52
Apply agent partition to load services and agent api (#16024)
* Apply agent partition to load services and agent api

changelog
2023-01-20 12:59:26 -05:00
Derek Menteer bb6951f99d
Fix proxy-defaults incorrectly merging config on upstreams. (#16021) 2023-01-20 11:25:51 -06:00
Jeff Boruszak feacfb2886
docs: link fixes for Envoy proxy page (#16023)
* Link path fixes

* update

* Revert "update"

This reverts commit 6b3344481c501a2d8e1190e80977cc1bb7ae7ee1.

* Link fixes
2023-01-20 11:12:18 -06:00
Ashwin Venkatesh 9f0a6c50b8
Add support for envoy readiness flags (#16015)
* Add support for envoy readiness flags
- add flags 'envoy-ready-bind-port` and `envoy-ready-bind-addr` on consul connect envoy to create a ready listener on that address.
2023-01-19 16:54:11 -05:00
am-ak ec3ebec505
Major updates and reorganizing of checks.mdx (#15806)
* Major updates and reorganizing of checks.mdx

* Update checks.mdx

Additional suggestion for clarity around gRPC `:/service-identifier` example

Signed-off-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>

* Update website/content/docs/discovery/checks.mdx

Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>

* Update website/content/docs/discovery/checks.mdx

Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>

* Update website/content/docs/discovery/checks.mdx

Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>

* Update website/content/docs/discovery/checks.mdx

Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>

* Update website/content/docs/discovery/checks.mdx

Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>

* Update website/content/docs/discovery/checks.mdx

Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>

* Update website/content/docs/discovery/checks.mdx

Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>

* Update website/content/docs/discovery/checks.mdx

Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>

* Update website/content/docs/discovery/checks.mdx

Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>

* Update website/content/docs/discovery/checks.mdx

Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>

* Update website/content/docs/discovery/checks.mdx

Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>

* Update website/content/docs/discovery/checks.mdx

Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>

Signed-off-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
2023-01-19 15:55:13 +00:00
John Murret acfc7452e9
Integration test for server rate limiting (#15960)
* rate limit test

* Have tests for the 3 modes

* added assertions for logs and metrics

* add comments to test sections

* add check for rate limit exceeded text in log assertion section.

* fix linting error

* updating test to use KV get and put.  move log assertion tolast.

* Adding logging for blocking messages in enforcing mode.  refactoring tests.

* modified test description

* formatting

* Apply suggestions from code review

Co-authored-by: Dan Upton <daniel@floppy.co>

* Update test/integration/consul-container/test/ratelimit/ratelimit_test.go

Co-authored-by: Dhia Ayachi <dhia@hashicorp.com>

* expand log checking so that it ensures both logs are they when they are supposed to be and not there when they are not expected to be.

* add retry on test

* Warn once when rate limit exceed regardless of enforcing vs permissive.

* Update test/integration/consul-container/test/ratelimit/ratelimit_test.go

Co-authored-by: Dan Upton <daniel@floppy.co>

Co-authored-by: Dan Upton <daniel@floppy.co>
Co-authored-by: Dhia Ayachi <dhia@hashicorp.com>
2023-01-19 08:43:33 -07:00
Thomas Eckert f198544270
Native API Gateway Config Entries (#15897)
* Stub Config Entries for Consul Native API Gateway (#15644)
* Add empty InlineCertificate struct and protobuf
* apigateway stubs
* Stub HTTPRoute in api pkg
* Stub HTTPRoute in structs pkg
* Simplify api.APIGatewayConfigEntry to be consistent w/ other entries
* Update makeConfigEntry switch, add docstring for HTTPRouteConfigEntry
* Add TCPRoute to MakeConfigEntry, return unique Kind
* Stub BoundAPIGatewayConfigEntry in agent
* Add RaftIndex to APIGatewayConfigEntry stub
* Add new config entry kinds to validation allow-list
* Add RaftIndex to other added config entry stubs
* Update usage metrics assertions to include new cfg entries
* Add Meta and acl.EnterpriseMeta to all new ConfigEntry types
* Remove unnecessary Services field from added config entry types
* Implement GetMeta(), GetEnterpriseMeta() for added config entry types
* Add meta field to proto, name consistently w/ existing config entries
* Format config_entry.proto
* Add initial implementation of CanRead + CanWrite for new config entry types
* Add unit tests for decoding of new config entry types
* Add unit tests for parsing of new config entry types
* Add unit tests for API Gateway config entry ACLs
* Return typed PermissionDeniedError on BoundAPIGateway CanWrite
* Add unit tests for added config entry ACLs
* Add BoundAPIGateway type to AllConfigEntryKinds
* Return proper kind from BoundAPIGateway
* Add docstrings for new config entry types
* Add missing config entry kinds to proto def
* Update usagemetrics_oss_test.go
* Use utility func for returning PermissionDeniedError
* EventPublisher subscriptions for Consul Native API Gateway (#15757)
* Create new event topics in subscribe proto
* Add tests for PBSubscribe func
* Make configs singular, add all configs to PBToStreamSubscribeRequest
* Add snapshot methods
* Add config_entry_events tests
* Add config entry kind to topic for new configs
* Add unit tests for snapshot methods
* Start adding integration test
* Test using the new controller code
* Update agent/consul/state/config_entry_events.go
* Check value of error
* Add controller stubs for API Gateway (#15837)
* update initial stub implementation
* move files, clean up mutex references
* Remove embed, use idiomatic names for constructors
* Remove stray file introduced in merge
* Add APIGateway validation (#15847)
* Add APIGateway validation
* Add additional validations
* Add cert ref validation
* Add protobuf definitions
* Fix up field types
* Add API structs
* Move struct fields around a bit
* APIGateway InlineCertificate validation (#15856)
* Add APIGateway validation
* Add additional validations
* Add protobuf definitions
* Tabs to spaces
* Add API structs
* Move struct fields around a bit
* Add validation for InlineCertificate
* Fix ACL test
* APIGateway BoundAPIGateway validation (#15858)
* Add APIGateway validation
* Add additional validations
* Add cert ref validation
* Add protobuf definitions
* Fix up field types
* Add API structs
* Move struct fields around a bit
* Add validation for BoundAPIGateway
* APIGateway TCPRoute validation (#15855)
* Add APIGateway validation
* Add additional validations
* Add cert ref validation
* Add protobuf definitions
* Fix up field types
* Add API structs
* Add TCPRoute normalization and validation
* Add forgotten Status
* Add some more field docs in api package
* Fix test
* Format imports
* Rename snapshot test variable names
* Add plumbing for Native API GW Subscriptions (#16003)

Co-authored-by: Sarah Alsmiller <sarah.alsmiller@hashicorp.com>
Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
Co-authored-by: sarahalsmiller <100602640+sarahalsmiller@users.noreply.github.com>
Co-authored-by: Andrew Stucki <andrew.stucki@hashicorp.com>
2023-01-18 22:14:34 +00:00
Anita Akaeze 66a88b65f2
NET-2038: Add envoy assertion function of listener verification (#15969) 2023-01-18 16:13:55 -05:00
Chris Thain 892ef07715
Support Vault agent auth config for AWS/GCP CA provider auth (#15970) 2023-01-18 11:53:04 -08:00
Derek Menteer 9b0984e5a6
Fix configuration merging for implicit tproxy upstreams. (#16000)
Fix configuration merging for implicit tproxy upstreams.

Change the merging logic so that the wildcard upstream has correct proxy-defaults
and service-defaults values combined into it. It did not previously merge all fields,
and the wildcard upstream did not exist unless service-defaults existed (it ignored
proxy-defaults, essentially).

Change the way we fetch upstream configuration in the xDS layer so that it falls back
to the wildcard when no matching upstream is found. This is what allows implicit peer
upstreams to have the correct "merged" config.

Change proxycfg to always watch local mesh gateway endpoints whenever a peer upstream
is found. This simplifies the logic so that we do not have to inspect the "merged"
configuration on peer upstreams to extract the mesh gateway mode.
2023-01-18 13:43:53 -06:00
Dan Upton 618deae657
xds: don't attempt to load-balance sessions for local proxies (#15789)
Previously, we'd begin a session with the xDS concurrency limiter
regardless of whether the proxy was registered in the catalog or in
the server's local agent state.

This caused problems for users who run `consul connect envoy` directly
against a server rather than a client agent, as the server's locally
registered proxies wouldn't be included in the limiter's capacity.

Now, the `ConfigSource` is responsible for beginning the session and we
only do so for services in the catalog.

Fixes: https://github.com/hashicorp/consul/issues/15753
2023-01-18 12:33:21 -06:00
Ashlee M Boyer e92392781e
[docs] Adjusting links for rewrite project (#15999) 2023-01-17 17:18:04 -08:00
Kendall Strautman 304f0e8cfd
chore: updates generated author docs (#15980) 2023-01-17 14:48:51 -08:00
Dan Stough dbe9d26962
chore(ci): fix compat ent compat tests for sidecars and gateways (#15997) 2023-01-17 17:16:55 -05:00
trujillo-adam 3fe381b507
fixes pre-devdot links in service defaults ref docs (#15989) 2023-01-17 10:09:14 -08:00
Chris S. Kim c262065276
Warn if ACL is enabled but no token is provided to Envoy (#15967) 2023-01-16 12:31:56 -05:00
Dhia Ayachi 9e99715ee8
avoid logging RPC errors when it's specific rate limiter errors (#15968)
* avoid logging RPC errors when it's specific rate limiter errors

* simplify if statements
2023-01-16 12:08:09 -05:00
Derek Menteer f926c7643f
Enforce lowercase peer names. (#15697)
Enforce lowercase peer names.

Prior to this change peer names could be mixed case.
This can cause issues, as peer names are used as DNS labels
in various locations. It also caused issues with envoy
configuration.
2023-01-13 14:20:28 -06:00
Ranjandas 8e9fe563fa
Update TG Docs with SAN match option when using SNI (#15971)
When using SNI in Terminating Gateway, Consul configures envoy to
have strict SAN matching. This requires all external services to
have SANs in their certificates and not having it will throw
CERTIFICATE_VERIFY_FAILED error.
2023-01-12 19:55:36 -08:00
Frank DiRocco 9420fd229b
Update go-discover to support ECS discovery (#13782)
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
2023-01-12 12:06:29 -06:00
Michael Wilkerson d224744f6a
removed Consul requirements from documentation as it may be confusing (#15958)
* removed Consul requirements from documentation as it may be confusing
2023-01-11 21:01:30 -08:00
R.B. Boyer 04673cb6e4
test: general cleanup and fixes for the container integration test suite (#15959)
- remove dep on consul main module
- use 'consul tls' subcommands instead of tlsutil
- use direct json config construction instead of agent/config structs
- merge libcluster and libagent packages together
- more widely use BuildContext
- get the OSS/ENT runner stuff working properly
- reduce some flakiness
- fix some correctness related to http/https API
2023-01-11 15:34:27 -06:00
Dan Stough 27d74de701
feat: add access logs to dataplane bootstrap rpc (#15951) 2023-01-11 13:40:09 -05:00
Chris Hut ad123d9564
ui/fix linting error and failing ui test (#15953)
* fix linting error

* Fix the failing version test as well
2023-01-11 10:48:04 -07:00
Dan Stough 797bfb1677
docs(access logs): new docs for access logging (#15948)
Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com>
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
2023-01-11 16:41:02 +00:00
Matt Keeler 554f1e6fee
Protobuf Modernization (#15949)
* Protobuf Modernization

Remove direct usage of golang/protobuf in favor of google.golang.org/protobuf

Marshallers (protobuf and json) needed some changes to account for different APIs.

Moved to using the google.golang.org/protobuf/types/known/* for the well known types including replacing some custom Struct manipulation with whats available in the structpb well known type package.

This also updates our devtools script to install protoc-gen-go from the right location so that files it generates conform to the correct interfaces.

* Fix go-mod-tidy make target to work on all modules
2023-01-11 09:39:10 -05:00
Paul Glass 1bf1686ebc
Add new config_file_service_registration token (#15828) 2023-01-10 10:24:02 -06:00
Dan Stough e0b7694a90
[OSS] feat: add access log config to consul envoy connect (#15946) 2023-01-09 15:16:00 -05:00
Chris S. Kim 82d6d12a13
Output user-friendly name for anonymous token (#15884) 2023-01-09 12:28:53 -06:00
Dan Upton f24f8c681f
Rate limit improvements and fixes (#15917)
- Fixes a panic when Operation.SourceAddr is nil (internal net/rpc calls)
- Adds proper HTTP response codes (429 and 503) for rate limit errors
- Makes the error messages clearer
- Enables automatic retries for rate-limit errors in the net/rpc stack
2023-01-09 10:20:05 +00:00
Semir Patel 2a90faa4b1
emit metrics for global rate limiting (#15891) 2023-01-06 17:49:33 -06:00
Jared Kirschner c088753e7b
docs: cluster peering upgrade clarification (#15935) 2023-01-06 18:30:10 -05:00
Thomas Eckert 8e3d0da54b
Update the README for the Consul API (#15936) 2023-01-06 21:10:56 +00:00
Derek Menteer a5915bd99f
Fix issue where TLS configuration was ignored for unix sockets in consul connect envoy. (#15913)
Fix issue where TLS configuration was ignored for unix sockets in consul connect envoy.
Disable xds check on bootstrap mode and change check to warn only.
2023-01-06 12:34:49 -06:00
Dhia Ayachi f17bc5ed73
inject logger and create logdrop sink (#15822)
* inject logger and create logdrop sink

* init sink with an empty struct instead of nil

* wrap a logger instead of a sink and add a discard logger to avoid double logging

* fix compile errors

* fix linter errors

* Fix bug where log arguments aren't properly formatted

* Move log sink construction outside of handler

* Add prometheus definition and docs for log drop counter

Co-authored-by: Daniel Upton <daniel@floppy.co>
2023-01-06 11:33:53 -07:00
Eric Haberkorn 01a0142d1f
Add the Lua Envoy extension (#15906) 2023-01-06 12:13:40 -05:00
Jared Kirschner a8840e710e
docs: fix markdown heading levels (#15912) 2023-01-06 11:37:19 -05:00
Matt Keeler 482c9e1122
Fix usage of protoc-gen-go-binary (#15920)
* Fix usage of protoc-gen-go-binary
* Pin protoc-gen-go-binary version
2023-01-06 11:30:54 -05:00
Tu Nguyen 9d1f030980
Update example link for Consul K8s NGINX ingress controller (#15916) 2023-01-06 08:11:13 -08:00
lbausch 5199137a6b
docs: Fix link (#14890)
Co-authored-by: David Yu <dyu@hashicorp.com>
2023-01-06 05:21:56 +00:00
claire labry 70f06e71f5
Onboard consul to use new version/VERSION file and reproducible actions (#15631)
* Onboard consul to use new .release/VERSION file and reproducible actions-go-build

* Onboard consul to use new .release/VERSION file and reproducible actions

* Onboard consul to use new .release/VERSION file and reproducible actions

* fix to consul

* Onboard consul to use new .release/VERSION file and reproducible actions

* Onboard consul to use new .release/VERSION file and reproducible actions

* Onboard consul to use new .release/VERSION file and reproducible actions

* test out ent changes

* just or testing

* Added setup go for build ui

* try removing VERSION file out of .release dir

* add checkout action for build ui and update checkout version

* try no -dev marker

* try removing extra ldflags

* test version

* add back in setup-go step?

* Update utils.js

read from static VERSION file

* remove actions-setup go

* add 1.15.0-dev

* Using prepare workflow for pre-stable channel workflow

* Test prepare workflow

* Remove set-product-version branch from release pipeline

* Use METADATA in environment

* Correct env vars

* Remove current branch from build trigger list

Co-authored-by: emilymianeil <emilymianeil@gmail.com>
Co-authored-by: Sarah <sthompson@hashicorp.com>
Co-authored-by: hc-github-team-nomad-core <github-team-nomad-core@hashicorp.com>
Co-authored-by: emily neil <63985869+emilymianeil@users.noreply.github.com>
2023-01-05 12:16:47 -08:00
Paul Glass a36839d9c3
Fix TLS_BadVerify test assertions on macOS (#15903) 2023-01-05 11:47:45 -06:00
Dan Upton 76fea384a3
grpc/acl: fix bug where ACL token was required even if disabled (#15904)
Fixes a bug introduced by #15346 where we'd always require an ACL
token even if ACLs were disabled because we were erroneously
treating `nil` identity as anonymous.
2023-01-05 16:31:18 +00:00
Dan Upton a36173e2b5
Fix missing comment in generated enterprise rate-limit file (#15895)
Fixes bug in #15564 where gofmt would strip out the generated code
warning comment because it was on the same line as the build tag.
2023-01-05 10:31:14 +00:00
Dan Upton 15c7c03fa5
grpc: switch servers and retry on error (#15892)
This is the OSS portion of enterprise PR 3822.

Adds a custom gRPC balancer that replicates the router's server cycling
behavior. Also enables automatic retries for RESOURCE_EXHAUSTED errors,
which we now get for free.
2023-01-05 10:21:27 +00:00
Nick Irvine 2c37b0afd1
fix: return error when config file with unknown extension is passed (#15107) 2023-01-04 16:57:00 -08:00
Tu Nguyen 39e6ca3bf6
Update api gateway version to latest (#15197)
* Update api gateway version to latest

* Update website/content/docs/api-gateway/install.mdx

* update to latest apigw version 0.5.1

* update consul and helm version
2023-01-04 12:44:02 -08:00
cskh bb797ff36c
Refactoring the peering integ test to accommodate coming changes of o… (#15885)
* Refactoring the peering integ test to accommodate coming changes of other upgrade scenarios.

- Add a utils package under test that contains methods to set up various test scenarios.
- Deduplication: have a single CreatingPeeringClusterAndSetup replace
  CreatingAcceptingClusterAndSetup and CreateDialingClusterAndSetup.
- Separate peering cluster creation and server registration.

* Apply suggestions from code review

Co-authored-by: Dan Stough <dan.stough@hashicorp.com>
2023-01-04 15:28:15 -05:00