* First cluster grpc service should be NodePort
This is based on the issue opened here https://github.com/hashicorp/consul-k8s/issues/1903
If you follow the documentation https://developer.hashicorp.com/consul/docs/k8s/deployment-configurations/single-dc-multi-k8s exactly as it is, the first cluster will only create the consul UI service on NodePort but not the rest of the services (including for grpc). By default, from the helm chart, they are created as headless services by setting clusterIP None. This will cause an issue for the second cluster to discover consul server on the first cluster over gRPC as it cannot simply cannot through gRPC default port 8502 and it ends up in an error as shown in the issue https://github.com/hashicorp/consul-k8s/issues/1903
As a solution, the grpc service should be exposed using NodePort (or LoadBalancer). I added those changes required in both cluster1-values.yaml and cluster2-values.yaml, and also a description for those changes for the normal users to understand. Kindly review and I hope this PR will be accepted.
* Update website/content/docs/k8s/deployment-configurations/single-dc-multi-k8s.mdx
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* Update website/content/docs/k8s/deployment-configurations/single-dc-multi-k8s.mdx
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* Update website/content/docs/k8s/deployment-configurations/single-dc-multi-k8s.mdx
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
---------
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
Co-authored-by: Ashvitha Sridharan <ashvitha.sridharan@hashicorp.com>
Co-authored-by: Freddy <freddygv@users.noreply.github.com>
Add a new envoy flag: "envoy_hcp_metrics_bind_socket_dir", a directory
where a unix socket will be created with the name
`<namespace>_<proxy_id>.sock` to forward Envoy metrics.
If set, this will configure:
- In bootstrap configuration a local stats_sink and static cluster.
These will forward metrics to a loopback listener sent over xDS.
- A dynamic listener listening at the socket path that the previously
defined static cluster is sending metrics to.
- A dynamic cluster that will forward traffic received at this listener
to the hcp-metrics-collector service.
Reasons for having a static cluster pointing at a dynamic listener:
- We want to secure the metrics stream using TLS, but the stats sink can
only be defined in bootstrap config. With dynamic listeners/clusters
we can use the proxy's leaf certificate issued by the Connect CA,
which isn't available at bootstrap time.
- We want to intelligently route to the HCP collector. Configuring its
addreess at bootstrap time limits our flexibility routing-wise. More
on this below.
Reasons for defining the collector as an upstream in `proxycfg`:
- The HCP collector will be deployed as a mesh service.
- Certificate management is taken care of, as mentioned above.
- Service discovery and routing logic is automatically taken care of,
meaning that no code changes are required in the xds package.
- Custom routing rules can be added for the collector using discovery
chain config entries. Initially the collector is expected to be
deployed to each admin partition, but in the future could be deployed
centrally in the default partition. These config entries could even be
managed by HCP itself.
* fixes for unsupported partitions field in CRD metadata block
* Apply suggestions from code review
Co-authored-by: Luke Kysow <1034429+lkysow@users.noreply.github.com>
---------
Co-authored-by: Luke Kysow <1034429+lkysow@users.noreply.github.com>
* Update the consul-k8s cli docs for the new `proxy log` subcommand
* Updated consul-k8s docs from PR feedback
* Added proxy log command to release notes
Updated Params field to re-frame as supporting arguments specific to the
supported vault-agent auth-auth methods with links to each methods
"#configuration" section.
Included a call out limits on parameters supported.
* Leverage ServiceResolver ConnectTimeout for route timeouts to make TerminatingGateway upstream timeouts configurable
* Regenerate golden files
* Add RequestTimeout field
* Add changelog entry
* converted main services page to services overview page
* set up services usage dirs
* added Define Services usage page
* converted health checks everything page to Define Health Checks usage page
* added Register Services and Nodes usage page
* converted Query with DNS to Discover Services and Nodes Overview page
* added Configure DNS Behavior usage page
* added Enable Static DNS Lookups usage page
* added the Enable Dynamic Queries DNS Queries usage page
* added the Configuration dir and overview page - may not need the overview, tho
* fixed the nav from previous commit
* added the Services Configuration Reference page
* added Health Checks Configuration Reference page
* updated service defaults configuraiton entry to new configuration ref format
* fixed some bad links found by checker
* more bad links found by checker
* another bad link found by checker
* converted main services page to services overview page
* set up services usage dirs
* added Define Services usage page
* converted health checks everything page to Define Health Checks usage page
* added Register Services and Nodes usage page
* converted Query with DNS to Discover Services and Nodes Overview page
* added Configure DNS Behavior usage page
* added Enable Static DNS Lookups usage page
* added the Enable Dynamic Queries DNS Queries usage page
* added the Configuration dir and overview page - may not need the overview, tho
* fixed the nav from previous commit
* added the Services Configuration Reference page
* added Health Checks Configuration Reference page
* updated service defaults configuraiton entry to new configuration ref format
* fixed some bad links found by checker
* more bad links found by checker
* another bad link found by checker
* fixed cross-links between new topics
* updated links to the new services pages
* fixed bad links in scale file
* tweaks to titles and phrasing
* fixed typo in checks.mdx
* started updating the conf ref to latest template
* update SD conf ref to match latest CT standard
* Apply suggestions from code review
Co-authored-by: Eddie Rowe <74205376+eddie-rowe@users.noreply.github.com>
* remove previous version of the checks page
* fixed cross-links
* Apply suggestions from code review
Co-authored-by: Eddie Rowe <74205376+eddie-rowe@users.noreply.github.com>
---------
Co-authored-by: Eddie Rowe <74205376+eddie-rowe@users.noreply.github.com>
* Update ingress-gateways.mdx
Added an example of running the HELM install for the ingress gateways using values.yaml
* Apply suggestions from code review
* Update ingress-gateways.mdx
Adds closing back ticks on example command. The suggesting UI strips them out.
---------
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* Add WAL documentation. Also fix some minor metrics registration details
* Add tests to verify metrics are registered correctly
* refactor and move wal docs
* Updates to the WAL overview page
* updates to enable WAL usage topic
* updates to the monitoring WAL backend topic
* updates for revert WAL topic
* a few tweaks to overview and udpated metadescriptions
* Apply suggestions from code review
Co-authored-by: Paul Banks <pbanks@hashicorp.com>
* make revert docs consistent with enable
* Apply suggestions from code review
Co-authored-by: Paul Banks <pbanks@hashicorp.com>
* address feedback
* address final feedback
* Apply suggestions from code review
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
---------
Co-authored-by: Paul Banks <pbanks@hashicorp.com>
Co-authored-by: trujillo-adam <ajosetru@gmail.com>
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
* Create empty files
* Copy over content for overview
* Copy over content for usage
* Copy over content for api-gateway config
* Copy over content for http-route config
* Copy over content for tcp-route config
* Copy over content for inline-certificate config
* Add docs to the sidebar
* Clean up overview. Start cleaning up usage
* Add BETA badge to API Gateways portion of nav
* Fix header
* Fix up usage
* Fix up API Gateway config
* Update paths to be consistent w/ other gateway docs
* Fix up http-route
* Fix up inline-certificate
* rename path
* Fix up tcp-route
* Add CodeTabs
* Add headers to config pages
* Fix configuration model for http route and inline certificate
* Add version callout to API gateway overview page
* Fix values for inline certificate
* Fix values for api gateway configuration
* Fix values for TCP Route config
* Fix values for HTTP Route config
* Adds link from k8s gateway to vm gateway page
* Remove versioning warning
* Serve overview page at ../api-gateway, consistent w/ mesh-gateway
* Remove weight field from tcp-route docs
* Linking to usage instead of overview from k8s api-gateway to vm api-gateway
* Fix issues in usage page
* Fix links in usage
* Capitalize Kubernetes
* Apply suggestions from code review
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* remove optional callout
* Apply suggestions from code review
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* Apply suggestions from code review
* Update website/content/docs/connect/gateways/api-gateway/configuration/api-gateway.mdx
* Fix formatting of Hostnames
* Update website/content/docs/api-gateway/index.mdx
* Update website/content/docs/connect/gateways/api-gateway/configuration/http-route.mdx
Co-authored-by: Andrew Stucki <andrew.stucki@hashicorp.com>
* Add cross-linking of config entries
* Fix rendering error on new operator usage docs
* Update website/content/docs/connect/gateways/api-gateway/configuration/http-route.mdx
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* Update website/content/docs/connect/gateways/api-gateway/configuration/http-route.mdx
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* Apply suggestions from code review
* Apply suggestions from code review
* Add BETA badges to config entry links
* http route updates
* Add Enterprise keys
* Use map instead of list for meta field, use consistent formatting
* Convert spaces to tabs
* Add all Enterprise info to TCP Route
* Use pascal case for JSON api-gateway example
* Add enterprise to HCL api-gw cfg
* Use pascal case for missed JSON config fields
* Add enterprise to JSON api-gw cfg
* Add enterprise to api-gw values
* adds enterprise to http route
* Update website/content/docs/connect/gateways/api-gateway/index.mdx
Co-authored-by: danielehc <40759828+danielehc@users.noreply.github.com>
* Add enterprise to api-gw spec
* Add missing namespace, partition + meta to specification
* fixes for http route
* Fix ordering of API Gatetway cfg spec items
* whitespace
* Add linking of values to tcp
* Apply suggestions from code review
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
* Fix comma in wrong place
* Apply suggestions from code review
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
* Move Certificates down
* Apply suggestions from code review
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
* Tabs to spaces in httproute
* Use configuration entry instead of config entry
* Fix indentations on api-gateway and tcp-route
* Add whitespace between code block and prose
* Apply suggestions from code review
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* adds <> to http route
---------
Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
Co-authored-by: Melisa Griffin <melisa.griffin@hashicorp.com>
Co-authored-by: Tu Nguyen <im2nguyen@users.noreply.github.com>
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
Co-authored-by: Tu Nguyen <im2nguyen@gmail.com>
Co-authored-by: Melisa Griffin <missylbytes@users.noreply.github.com>
Co-authored-by: Andrew Stucki <andrew.stucki@hashicorp.com>
Co-authored-by: danielehc <40759828+danielehc@users.noreply.github.com>
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
* Remove Consul Client installation option
With Consul-K8S 1.0 and introduction of Consul-Dataplane, K8S has
the option to run without running Consul Client agents.
* remove note referring to the same documentation
* Added instructions on the use of httpsPort when servers are not running TLS enabled
* Modified titile and description
* Update existing docs from Consul API Gateway -> API Gateway for Kubernetes
* Update page header to reflect page title change
* Update nav title to match new page title
* for tab testing
* updates
* Update
* adding sandbox to test conf ref types
* testing tweaks to the conf ref template
* reintroduce tabbed specification
* applied feedback from MKO session
* applied feedback on format from luke and jared
* Apply suggestions from code review
Co-authored-by: Dan Upton <daniel@floppy.co>
* fixed some minor HCL formatting in complete conf
* Apply suggestions from code review
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
* fixed bad link
* resolving conflicts
---------
Co-authored-by: boruszak <jeffrey.boruszak@hashicorp.com>
Co-authored-by: Dan Upton <daniel@floppy.co>
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
* remove legacy tokens
* remove lingering legacy token references from docs
* update language and naming for token secrets and accessor IDs
* updates all tokenID references to clarify accessorID
* remove token type references and lookup tokens by accessorID index
* remove unnecessary constants
* replace additional tokenID param names
* Add warning info for deprecated -id parameter
Co-authored-by: Paul Glass <pglass@hashicorp.com>
* Update field comment
Co-authored-by: Paul Glass <pglass@hashicorp.com>
---------
Co-authored-by: Paul Glass <pglass@hashicorp.com>
* doc: clarify the default time unit in service-resolver.ConnectTimeout
* Update website/content/docs/connect/config-entries/service-resolver.mdx
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
---------
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
The generate_lease=true configuration is unnecessary and generates a note about performance implications in Vault logs. Remove this configuration so that the default value of generate_lease=false is used instead.
* Document how numRetries can't be set to 0
Resolves https://github.com/hashicorp/consul/issues/11816 and https://github.com/hashicorp/consul/issues/8516.
* Update website/content/docs/connect/config-entries/service-router.mdx
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
---------
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* Update connect-internals.mdx
Removed most references for 'Connect' given the terminology has long been deprecated in official use.
* Apply suggestions from code review
Co-authored-by: Blake Covarrubias <blake@covarrubi.as>
* Update connect-internals.mdx
Updates based on Blakes recommendations
* Update connect-internals.mdx
---------
Co-authored-by: Blake Covarrubias <blake@covarrubi.as>
Co-authored-by: Kendall Strautman <36613477+kendallstrautman@users.noreply.github.com>
* remove legacy tokens
* Update test comment
Co-authored-by: Paul Glass <pglass@hashicorp.com>
* fix imports
* update docs for additional CLI changes
* add test case for anonymous token
* set deprecated api fields to json ignore and fix patch errors
* update changelog to breaking-change
* fix import
* update api docs to remove legacy reference
* fix docs nav data
---------
Co-authored-by: Paul Glass <pglass@hashicorp.com>
* Initial page and nav data
* Formatting
* Fixes
* Page description
* DNS lookup fixes
* admin partition link
* Control Plane Resiliency rephrase
* Dataplanes/xDS callout
* word choice correction
* Consul as Vault backend clarifications
* Link to blog post on testing
* Update website/content/docs/architecture/scale.mdx
Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com>
* Update website/content/docs/architecture/scale.mdx
* Apply suggestions from code review
Co-authored-by: Tu Nguyen <im2nguyen@users.noreply.github.com>
* Update website/content/docs/architecture/scale.mdx
* Update website/content/docs/architecture/scale.mdx
Co-authored-by: Tu Nguyen <im2nguyen@users.noreply.github.com>
Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com>
* Add support for envoy readiness flags
- add flags 'envoy-ready-bind-port` and `envoy-ready-bind-addr` on consul connect envoy to create a ready listener on that address.
* Stub Config Entries for Consul Native API Gateway (#15644)
* Add empty InlineCertificate struct and protobuf
* apigateway stubs
* Stub HTTPRoute in api pkg
* Stub HTTPRoute in structs pkg
* Simplify api.APIGatewayConfigEntry to be consistent w/ other entries
* Update makeConfigEntry switch, add docstring for HTTPRouteConfigEntry
* Add TCPRoute to MakeConfigEntry, return unique Kind
* Stub BoundAPIGatewayConfigEntry in agent
* Add RaftIndex to APIGatewayConfigEntry stub
* Add new config entry kinds to validation allow-list
* Add RaftIndex to other added config entry stubs
* Update usage metrics assertions to include new cfg entries
* Add Meta and acl.EnterpriseMeta to all new ConfigEntry types
* Remove unnecessary Services field from added config entry types
* Implement GetMeta(), GetEnterpriseMeta() for added config entry types
* Add meta field to proto, name consistently w/ existing config entries
* Format config_entry.proto
* Add initial implementation of CanRead + CanWrite for new config entry types
* Add unit tests for decoding of new config entry types
* Add unit tests for parsing of new config entry types
* Add unit tests for API Gateway config entry ACLs
* Return typed PermissionDeniedError on BoundAPIGateway CanWrite
* Add unit tests for added config entry ACLs
* Add BoundAPIGateway type to AllConfigEntryKinds
* Return proper kind from BoundAPIGateway
* Add docstrings for new config entry types
* Add missing config entry kinds to proto def
* Update usagemetrics_oss_test.go
* Use utility func for returning PermissionDeniedError
* EventPublisher subscriptions for Consul Native API Gateway (#15757)
* Create new event topics in subscribe proto
* Add tests for PBSubscribe func
* Make configs singular, add all configs to PBToStreamSubscribeRequest
* Add snapshot methods
* Add config_entry_events tests
* Add config entry kind to topic for new configs
* Add unit tests for snapshot methods
* Start adding integration test
* Test using the new controller code
* Update agent/consul/state/config_entry_events.go
* Check value of error
* Add controller stubs for API Gateway (#15837)
* update initial stub implementation
* move files, clean up mutex references
* Remove embed, use idiomatic names for constructors
* Remove stray file introduced in merge
* Add APIGateway validation (#15847)
* Add APIGateway validation
* Add additional validations
* Add cert ref validation
* Add protobuf definitions
* Fix up field types
* Add API structs
* Move struct fields around a bit
* APIGateway InlineCertificate validation (#15856)
* Add APIGateway validation
* Add additional validations
* Add protobuf definitions
* Tabs to spaces
* Add API structs
* Move struct fields around a bit
* Add validation for InlineCertificate
* Fix ACL test
* APIGateway BoundAPIGateway validation (#15858)
* Add APIGateway validation
* Add additional validations
* Add cert ref validation
* Add protobuf definitions
* Fix up field types
* Add API structs
* Move struct fields around a bit
* Add validation for BoundAPIGateway
* APIGateway TCPRoute validation (#15855)
* Add APIGateway validation
* Add additional validations
* Add cert ref validation
* Add protobuf definitions
* Fix up field types
* Add API structs
* Add TCPRoute normalization and validation
* Add forgotten Status
* Add some more field docs in api package
* Fix test
* Format imports
* Rename snapshot test variable names
* Add plumbing for Native API GW Subscriptions (#16003)
Co-authored-by: Sarah Alsmiller <sarah.alsmiller@hashicorp.com>
Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
Co-authored-by: sarahalsmiller <100602640+sarahalsmiller@users.noreply.github.com>
Co-authored-by: Andrew Stucki <andrew.stucki@hashicorp.com>
When using SNI in Terminating Gateway, Consul configures envoy to
have strict SAN matching. This requires all external services to
have SANs in their certificates and not having it will throw
CERTIFICATE_VERIFY_FAILED error.
Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com>
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
* inject logger and create logdrop sink
* init sink with an empty struct instead of nil
* wrap a logger instead of a sink and add a discard logger to avoid double logging
* fix compile errors
* fix linter errors
* Fix bug where log arguments aren't properly formatted
* Move log sink construction outside of handler
* Add prometheus definition and docs for log drop counter
Co-authored-by: Daniel Upton <daniel@floppy.co>
* Update api gateway version to latest
* Update website/content/docs/api-gateway/install.mdx
* update to latest apigw version 0.5.1
* update consul and helm version
* add functions for returning the max and min Envoy major versions
- added an UnsupportedEnvoyVersions list
- removed an unused error from TestDetermineSupportedProxyFeaturesFromString
- modified minSupportedVersion to use the function for getting the Min Envoy major version. Using just the major version without the patch is equivalent to using `.0`
* added a function for executing the envoy --version command
- added a new exec.go file to not be locked to unix system
* added envoy version check when using consul connect envoy
* added changelog entry
* added docs change
Melisa Griffin