* [OSS] Post Consul 1.16 updates (#17606)
* chore: update dev build to 1.17
* chore(ci): add nightly 1.16 test
Drop the oldest and add the newest running release branch to nightly
builds.
* Add writeAuditRPCEvent to agent_oss (#17607)
* Add writeAuditRPCEvent to agent_oss
* fix the other diffs
* backport change log
* Add Envoy and Consul version constraints to Envoy extensions (#17612)
* [API Gateway] Fix trust domain for external peered services in synthesis code (#17609)
* [API Gateway] Fix trust domain for external peered services in synthesis code
* Add changelog
* backport ent changes to oss (#17614)
* backport ent changes to oss
* Update .changelog/_5669.txt
Co-authored-by: Michael Zalimeni <michael.zalimeni@hashicorp.com>
---------
Co-authored-by: Michael Zalimeni <michael.zalimeni@hashicorp.com>
* Update intentions.mdx (#17619)
Make behaviour of L7 intentions clearer
* enterprise changelog update for audit (#17625)
* Update list of Envoy versions (#17546)
* [API Gateway] Fix rate limiting for API gateways (#17631)
* [API Gateway] Fix rate limiting for API gateways
* Add changelog
* Fix failing unit tests
* Fix operator usage tests for api package
* sort some imports that are wonky between oss and ent (#17637)
* PmTLS and tproxy improvements with failover and L7 traffic mgmt for k8s (#17624)
* porting over changes from enterprise repo to oss
* applied feedback on service mesh for k8s overview
* fixed typo
* removed ent-only build script file
* Apply suggestions from code review
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: David Yu <dyu@hashicorp.com>
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
---------
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
Co-authored-by: David Yu <dyu@hashicorp.com>
* Delete check-legacy-links-format.yml (#17647)
* docs: Reference doc updates for permissive mTLS settings (#17371)
* Reference doc updates for permissive mTLS settings
* Document config entry filtering
* Fix minor doc errors (double slashes in link url paths)
---------
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* Add generic experiments configuration and use it to enable catalog v2 resources (#17604)
* Add generic experiments configuration and use it to enable catalog v2 resources
* Run formatting with -s as CI will validate that this has been done
* api-gateway: stop adding all header filters to virtual host when generating xDS (#17644)
* Add header filter to api-gateway xDS golden test
* Stop adding all header filters to virtual host when generating xDS for api-gateway
* Regenerate xDS golden file for api-gateway w/ header filter
* fix: add agent info reporting log (#17654)
* Add new Consul 1.16 docs (#17651)
* Merge pull request #5773 from hashicorp/docs/rate-limiting-from-ip-addresses-1.16
updated docs for rate limiting for IP addresses - 1.16
* Merge pull request #5609 from hashicorp/docs/enterprise-utilization-reporting
Add docs for enterprise utilization reporting
* Merge pull request #5734 from hashicorp/docs/envoy-ext-1.16
Docs/envoy ext 1.16
* Merge pull request #5773 from hashicorp/docs/rate-limiting-from-ip-addresses-1.16
updated docs for rate limiting for IP addresses - 1.16
* Merge pull request #5609 from hashicorp/docs/enterprise-utilization-reporting
Add docs for enterprise utilization reporting
* Merge pull request #5734 from hashicorp/docs/envoy-ext-1.16
Docs/envoy ext 1.16
* fix build errors
---------
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* Default `ProxyType` for builtin extensions (#17657)
* Post 1.16.0-rc1 updates (#17663)
- Update changelog to include new entries from release
- Update submodule versions to latest published
* Update service-defaults.mdx (#17656)
* docs: Sameness Groups (#17628)
* port from enterprise branch
* Apply suggestions from code review
Co-authored-by: shanafarkas <105076572+shanafarkas@users.noreply.github.com>
* Update website/content/docs/connect/cluster-peering/usage/create-sameness-groups.mdx
* next steps
* Update website/content/docs/connect/cluster-peering/usage/create-sameness-groups.mdx
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* Update website/content/docs/k8s/connect/cluster-peering/usage/create-sameness-groups.mdx
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
---------
Co-authored-by: shanafarkas <105076572+shanafarkas@users.noreply.github.com>
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* Remove "BETA" marker from config entries (#17670)
* CAPIgw for K8s installation updates for 1.16 (#17627)
* trimmed CRD step and reqs from installation
* updated tech specs
* Apply suggestions from code review
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
Co-authored-by: Jeff Apple <79924108+Jeff-Apple@users.noreply.github.com>
* added upgrade instruction
* removed tcp port req
* described downtime and DT-less upgrades
* applied additional review feedback
---------
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
Co-authored-by: Jeff Apple <79924108+Jeff-Apple@users.noreply.github.com>
* additional feedback on API gateway upgrades (#17677)
* additional feedback
* Update website/content/docs/api-gateway/upgrades.mdx
Co-authored-by: Jeff Apple <79924108+Jeff-Apple@users.noreply.github.com>
---------
Co-authored-by: Jeff Apple <79924108+Jeff-Apple@users.noreply.github.com>
* docs: JWT Authorization for intentions (#17643)
* Initial page/nav creation
* configuration entry reference page
* Usage + fixes
* service intentions page
* usage
* description
* config entry updates
* formatting fixes
* Update website/content/docs/connect/config-entries/service-intentions.mdx
Co-authored-by: Paul Glass <pglass@hashicorp.com>
* service intentions review fixes
* Overview page review fixes
* Apply suggestions from code review
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
---------
Co-authored-by: Paul Glass <pglass@hashicorp.com>
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* docs: minor fixes to JWT auth docs (#17680)
* Fixes
* service intentions fixes
* Fix two WAL metrics in docs/agent/telemetry.mdx (#17593)
* updated failover for k8s w-tproxy page title (#17683)
* Add release notes 1.16 rc (#17665)
* Merge pull request #5773 from hashicorp/docs/rate-limiting-from-ip-addresses-1.16
updated docs for rate limiting for IP addresses - 1.16
* Merge pull request #5609 from hashicorp/docs/enterprise-utilization-reporting
Add docs for enterprise utilization reporting
* Merge pull request #5734 from hashicorp/docs/envoy-ext-1.16
Docs/envoy ext 1.16
* Add release notes for 1.16-rc
* Add consul-e license utlization reporting
* Update with rc absolute links
* Update with rc absolute links
* fix typo
* Apply suggestions from code review
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* Update to use callout component
* address typo
* docs: FIPS 140-2 Compliance (#17668)
* Page + nav + formatting
* link fix
* Update website/content/docs/enterprise/fips.mdx
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* Update website/content/docs/enterprise/fips.mdx
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* Update website/content/docs/enterprise/fips.mdx
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* Update website/content/docs/enterprise/fips.mdx
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* Update website/content/docs/enterprise/fips.mdx
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* Update website/content/docs/enterprise/fips.mdx
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* Update website/content/docs/enterprise/fips.mdx
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* Update website/content/docs/enterprise/fips.mdx
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* Update website/content/docs/enterprise/fips.mdx
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* Update website/content/docs/enterprise/fips.mdx
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* link fix
* Apply suggestions from code review
Co-authored-by: Jeff Apple <79924108+Jeff-Apple@users.noreply.github.com>
* Update website/content/docs/enterprise/fips.mdx
---------
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
Co-authored-by: Jeff Apple <79924108+Jeff-Apple@users.noreply.github.com>
* fix apigw install values file
* fix typos in release notes
---------
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
Co-authored-by: Jeff Apple <79924108+Jeff-Apple@users.noreply.github.com>
* fix release notes links (#17687)
* adding redirects for tproxy and envoy extensions (#17688)
* adding redirects
* Apply suggestions from code review
* Fix FIPS copy (#17691)
* fix release notes links
* fix typos on fips docs
* [NET-4107][Supportability] Log Level set to TRACE and duration set to 5m for consul-debug (#17596)
* changed duration to 5 mins and log level to trace
* documentation update
* change log
* ENT merge of ext-authz extension updates (#17684)
* docs: Update default values for Envoy extension proxy types (#17676)
* fix: stop peering delete routine on leader loss (#17483)
* Refactor disco chain prioritize by locality structs (#17696)
This includes prioritize by localities on disco chain targets rather than
resolvers, allowing different targets within the same partition to have
different policies.
* agent: remove agent cache dependency from service mesh leaf certificate management (#17075)
* agent: remove agent cache dependency from service mesh leaf certificate management
This extracts the leaf cert management from within the agent cache.
This code was produced by the following process:
1. All tests in agent/cache, agent/cache-types, agent/auto-config,
agent/consul/servercert were run at each stage.
- The tests in agent matching .*Leaf were run at each stage.
- The tests in agent/leafcert were run at each stage after they
existed.
2. The former leaf cert Fetch implementation was extracted into a new
package behind a "fake RPC" endpoint to make it look almost like all
other cache type internals.
3. The old cache type was shimmed to use the fake RPC endpoint and
generally cleaned up.
4. I selectively duplicated all of Get/Notify/NotifyCallback/Prepopulate
from the agent/cache.Cache implementation over into the new package.
This was renamed as leafcert.Manager.
- Code that was irrelevant to the leaf cert type was deleted
(inlining blocking=true, refresh=false)
5. Everything that used the leaf cert cache type (including proxycfg
stuff) was shifted to use the leafcert.Manager instead.
6. agent/cache-types tests were moved and gently replumbed to execute
as-is against a leafcert.Manager.
7. Inspired by some of the locking changes from derek's branch I split
the fat lock into N+1 locks.
8. The waiter chan struct{} was eventually replaced with a
singleflight.Group around cache updates, which was likely the biggest
net structural change.
9. The awkward two layers or logic produced as a byproduct of marrying
the agent cache management code with the leaf cert type code was
slowly coalesced and flattened to remove confusion.
10. The .*Leaf tests from the agent package were copied and made to work
directly against a leafcert.Manager to increase direct coverage.
I have done a best effort attempt to port the previous leaf-cert cache
type's tests over in spirit, as well as to take the e2e-ish tests in the
agent package with Leaf in the test name and copy those into the
agent/leafcert package to get more direct coverage, rather than coverage
tangled up in the agent logic.
There is no net-new test coverage, just coverage that was pushed around
from elsewhere.
* [core]: Pin github action workflows (#17695)
* docs: missing changelog for _5517 (#17706)
* add enterprise notes for IP-based rate limits (#17711)
* add enterprise notes for IP-based rate limits
* Apply suggestions from code review
Co-authored-by: Tu Nguyen <im2nguyen@users.noreply.github.com>
Co-authored-by: David Yu <dyu@hashicorp.com>
* added bolded 'Enterprise' in list items.
---------
Co-authored-by: Tu Nguyen <im2nguyen@users.noreply.github.com>
Co-authored-by: David Yu <dyu@hashicorp.com>
* Update compatibility.mdx (#17713)
* Remove extraneous version info for Config entries (#17716)
* Update terminating-gateway.mdx
* Update exported-services.mdx
* Update mesh.mdx
* fix: typo in link to section (#17527)
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* Bump Alpine to 3.18 (#17719)
* Update Dockerfile
* Create 17719.txt
* NET-1825: New ACL token creation docs (#16465)
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com>
* [NET-3865] [Supportability] Additional Information in the output of 'consul operator raft list-peers' (#17582)
* init
* fix tests
* added -detailed in docs
* added change log
* fix doc
* checking for entry in map
* fix tests
* removed detailed flag
* removed detailed flag
* revert unwanted changes
* removed unwanted changes
* updated change log
* pr review comment changes
* pr comment changes single API instead of two
* fix change log
* fix tests
* fix tests
* fix test operator raft endpoint test
* Update .changelog/17582.txt
Co-authored-by: Semir Patel <semir.patel@hashicorp.com>
* nits
* updated docs
---------
Co-authored-by: Semir Patel <semir.patel@hashicorp.com>
* OSS merge: Update error handling login when applying extensions (#17740)
* Bump atlassian/gajira-transition from 3.0.0 to 3.0.1 (#17741)
Bumps [atlassian/gajira-transition](https://github.com/atlassian/gajira-transition) from 3.0.0 to 3.0.1.
- [Release notes](https://github.com/atlassian/gajira-transition/releases)
- [Commits](4749176faf...38fc9cd61b)
---
updated-dependencies:
- dependency-name: atlassian/gajira-transition
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Add truncation to body (#17723)
* docs: Failover overview minor fix (#17743)
* Incorrect symbol
* Clarification
* slight edit for clarity
* docs - update Envoy and Dataplane compat matrix (#17752)
* Update envoy.mdx
added more detail around default versus other compatible versions
* validate localities on agent configs and registration endpoints (#17712)
* Updated docs added explanation. (#17751)
* init
* fix tests
* added -detailed in docs
* added change log
* fix doc
* checking for entry in map
* fix tests
* removed detailed flag
* removed detailed flag
* revert unwanted changes
* removed unwanted changes
* updated change log
* pr review comment changes
* pr comment changes single API instead of two
* fix change log
* fix tests
* fix tests
* fix test operator raft endpoint test
* Update .changelog/17582.txt
Co-authored-by: Semir Patel <semir.patel@hashicorp.com>
* nits
* updated docs
* explanation added
---------
Co-authored-by: Semir Patel <semir.patel@hashicorp.com>
* Update index.mdx (#17749)
* added redirects and updated links (#17764)
* Add transparent proxy enhancements changelog (#17757)
* docs - remove use of consul leave during upgrade instructions (#17758)
* Fix issue with streaming service health watches. (#17775)
Fix issue with streaming service health watches.
This commit fixes an issue where the health streams were unaware of service
export changes. Whenever an exported-services config entry is modified, it is
effectively an ACL change.
The bug would be triggered by the following situation:
- no services are exported
- an upstream watch to service X is spawned
- the streaming backend filters out data for service X (due to lack of exports)
- service X is finally exported
In the situation above, the streaming backend does not trigger a refresh of its
data. This means that any events that were supposed to have been received prior
to the export are NOT backfilled, and the watches never see service X spawning.
We currently have decided to not trigger a stream refresh in this situation due
to the potential for a thundering herd effect (touching exports would cause a
re-fetch of all watches for that partition, potentially). Therefore, a local
blocking-query approach was added by this commit for agentless.
It's also worth noting that the streaming subscription is currently bypassed
most of the time with agentful, because proxycfg has a `req.Source.Node != ""`
which prevents the `streamingEnabled` check from passing. This means that while
agents should technically have this same issue, they don't experience it with
mesh health watches.
Note that this is a temporary fix that solves the issue for proxycfg, but not
service-discovery use cases.
* Property Override validation improvements (#17759)
* Reject inbound Prop Override patch with Services
Services filtering is only supported for outbound TrafficDirection patches.
* Improve Prop Override unexpected type validation
- Guard against additional invalid parent and target types
- Add specific error handling for Any fields (unsupported)
* Fixes (#17765)
* Update license get explanation (#17782)
This PR is to clarify what happens if the license get command is run on a follower if the leader hasn't been updated with a newer license.
* Add Patch index to Prop Override validation errors (#17777)
When a patch is found invalid, include its index for easier debugging
when multiple patches are provided.
* Stop referenced jwt providers from being deleted (#17755)
* Stop referenced jwt providers from being deleted
* Implement a Catalog Controllers Lifecycle Integration Test (#17435)
* Implement a Catalog Controllers Lifecycle Integration Test
* Prevent triggering the race detector.
This allows defining some variables for protobuf constants and using those in comparisons. Without that, something internal in the fmt package ended up looking at the protobuf message size cache and triggering the race detector.
* HCP Add node id/name to config (#17750)
* Catalog V2 Container Based Integration Test (#17674)
* Implement the Catalog V2 controller integration container tests
This now allows the container tests to import things from the root module. However for now we want to be very restrictive about which packages we allow importing.
* Add an upgrade test for the new catalog
Currently this should be dormant and not executed. However its put in place to detect breaking changes in the future and show an example of how to do an upgrade test with integration tests structured like catalog v2.
* Make testutil.Retry capable of performing cleanup operations
These cleanup operations are executed after each retry attempt.
* Move TestContext to taking an interface instead of a concrete testing.T
This allows this to be used on a retry.R or generally anything that meets the interface.
* Move to using TestContext instead of background contexts
Also this forces all test methods to implement the Cleanup method now instead of that being an optional interface.
Co-authored-by: Daniel Upton <daniel@floppy.co>
* Fix Docs for Trails Leader By (#17763)
* init
* fix tests
* added -detailed in docs
* added change log
* fix doc
* checking for entry in map
* fix tests
* removed detailed flag
* removed detailed flag
* revert unwanted changes
* removed unwanted changes
* updated change log
* pr review comment changes
* pr comment changes single API instead of two
* fix change log
* fix tests
* fix tests
* fix test operator raft endpoint test
* Update .changelog/17582.txt
Co-authored-by: Semir Patel <semir.patel@hashicorp.com>
* nits
* updated docs
* explanation added
* fix doc
* fix docs
---------
Co-authored-by: Semir Patel <semir.patel@hashicorp.com>
* Improve Prop Override docs examples (#17799)
- Provide more realistics examples for setting properties not already
supported natively by Consul
- Remove superfluous commas from HCL, correct target service name, and
fix service defaults vs. proxy defaults in examples
- Align existing integration test to updated docs
* Test permissive mTLS filter chain not configured with tproxy disabled (#17747)
* Add documentation for remote debugging of integration tests. (#17800)
* Add documentation for remote debugging of integration tests.
* add link from main docs page.
* changes related to PR feedback
* Clarify limitations of Prop Override extension (#17801)
Explicitly document the limitations of the extension, particularly what
kind of fields it is capable of modifying.
* Fix formatting for webhook-certs Consul tutorial (#17810)
* Fix formatting for webhook-certs Consul tutorial
* Make a small grammar change to also pick up whitespace changes necessary for formatting
---------
Co-authored-by: David Yu <dyu@hashicorp.com>
* Add jwt-authn metrics to jwt-provider docs (#17816)
* [NET-3095] add jwt-authn metrics docs
* Change URLs for redirects from RC to default latest (#17822)
* Set GOPRIVATE for all hashicorp repos in CI (#17817)
Consistently set GOPRIVATE to include all hashicorp repos, s.t. private
modules are successfully pulled in enterprise CI.
* Make locality aware routing xDS changes (#17826)
* Fixup consul-container/test/debugging.md (#17815)
Add missing `-t` flag and fix minor typo.
* fixes#17732 - AccessorID in request body should be optional when updating ACL token (#17739)
* AccessorID in request body should be optional when updating ACL token
* add a test case
* fix test case
* add changelog entry for PR #17739
* CA provider doc updates and Vault provider minor update (#17831)
Update CA provider docs
Clarify that providers can differ between
primary and secondary datacenters
Provide a comparison chart for consul vs
vault CA providers
Loosen Vault CA provider validation for RootPKIPath
Update Vault CA provider documentation
* ext-authz Envoy extension: support `localhost` as a valid target URI. (#17821)
* CI Updates (#17834)
* Ensure that git access to private repos uses the ELEVATED_GITHUB_TOKEN
* Bump the runner size for the protobuf generation check
This has failed previously when the runner process that communicates with GitHub gets starved causing the job to fail.
* counter part of ent pr (#17618)
* watch: support -filter for consul watch: checks, services, nodes, service (#17780)
* watch: support -filter for watch checks
* Add filter for watch nodes, services, and service
- unit test added
- Add changelog
- update doc
* Trigger OSS => ENT merge for all release branches (#17853)
Previously, this only triggered for release/*.*.x branches; however, our release process involves cutting a release/1.16.0 branch, for example, at time of code freeze these days. Any PRs to that branch after code freeze today do not make their way to consul-enterprise. This will make behavior for a .0 branch consistent with current behavior for a .x branch.
* Update service-mesh.mdx (#17845)
Deleted two commas which looks quite like some leftovers.
* Add docs for sameness groups with resolvers. (#17851)
* docs: add note about path prefix matching behavior for HTTPRoute config (#17860)
* Add note about path prefix matching behavior for HTTPRoute config
* Update website/content/docs/connect/gateways/api-gateway/configuration/http-route.mdx
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
---------
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* docs: update upgrade to consul-dataplane docs on k8s (#17852)
* resource: add `AuthorizerContext` helper method (#17393)
* resource: enforce consistent naming of resource types (#17611)
For consistency, resource type names must follow these rules:
- `Group` must be snake case, and in most cases a single word.
- `GroupVersion` must be lowercase, start with a "v" and end with a number.
- `Kind` must be pascal case.
These were chosen because they map to our protobuf type naming
conventions.
* tooling: generate protoset file (#17364)
Extends the `proto` make target to generate a protoset file for use with
grpcurl etc.
* Fix a bug that wrongly trims domains when there is an overlap with DC name (#17160)
* Fix a bug that wrongly trims domains when there is an overlap with DC name
Before this change, when DC name and domain/alt-domain overlap, the domain name incorrectly trimmed from the query.
Example:
Given: datacenter = dc-test, alt-domain = test.consul.
Querying for "test-node.node.dc-test.consul" will faile, because the
code was trimming "test.consul" instead of just ".consul"
This change, fixes the issue by adding dot (.) before trimming
* trimDomain: ensure domain trimmed without modyfing original domains
* update changelog
---------
Co-authored-by: Dhia Ayachi <dhia@hashicorp.com>
* deps: aws-sdk-go v1.44.289 (#17876)
Signed-off-by: Dan Bond <danbond@protonmail.com>
* api-gateway: add operation cannot be fulfilled error to common errors (#17874)
* add error message
* Update website/content/docs/api-gateway/usage/errors.mdx
Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
* fix formating issues
---------
Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
* api-gateway: add step to upgrade instructions for creating intentions (#17875)
* Changelog - add 1.13.9, 1.14.8, and 1.15.4 (#17889)
* docs: update config enable_debug (#17866)
* update doc for config enable_debug
* Update website/content/docs/agent/config/config-files.mdx
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
---------
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* Update wording on WAN fed and intermediate_pki_path (#17850)
* Allow service identity tokens the ability to read jwt-providers (#17893)
* Allow service identity tokens the ability to read jwt-providers
* more tests
* service_prefix tests
* Update docs (#17476)
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* Add emit_tags_as_labels to envoy bootstrap config when using Consul Telemetry Collector (#17888)
* Fix command from kg to kubectl get (#17903)
* Create and update release notes for 1.16 and 1.2 (#17895)
* update release notes for 1.16 and 1.2
* update latest consul core release
* Propose new changes to APIgw upgrade instructions (#17693)
* Propose new changes to APIgw upgrade instructions
* fix build error
* update callouts to render correctly
* Add hideClipboard to log messages
* Added clarification around consul k8s and crds
* Add workflow to verify linux release packages (#17904)
* adding docker files to verify linux packages.
* add verifr-release-linux.yml
* updating name
* pass inputs directly into jobs
* add other linux package platforms
* remove on push
* fix TARGETARCH on debian and ubuntu so it can check arm64 and amd64
* fixing amazon to use the continue line
* add ubuntu i386
* fix comment lines
* working
* remove commented out workflow jobs
* Apply suggestions from code review
Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
* update fedora and ubuntu to use latest tag
---------
Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
* Reference hashicorp/consul instead of consul for Docker image (#17914)
* Reference hashicorp/consul instead of consul for Docker image
* Update Make targets that pull consul directly
* Update Consul K8s Upgrade Doc Updates (#17921)
Updating upgrade procedures to encompass expected errors during upgrade process from v1.13.x to v1.14.x.
* Update sameness-group.mdx (#17915)
* Update create-sameness-groups.mdx (#17927)
* deps: coredns v1.10.1 (#17912)
* Ensure RSA keys are at least 2048 bits in length (#17911)
* Ensure RSA keys are at least 2048 bits in length
* Add changelog
* update key length check for FIPS compliance
* Fix no new variables error and failing to return when error exists from
validating
* clean up code for better readability
* actually return value
* tlsutil: Fix check TLS configuration (#17481)
* tlsutil: Fix check TLS configuration
* Rewording docs.
* Update website/content/docs/services/configuration/checks-configuration-reference.mdx
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* Fix typos and add changelog entry.
---------
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* docs: Deprecations for connect-native SDK and specific connect native APIs (#17937)
* Update v1_16_x.mdx
* Update connect native golang page
---------
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* Revert "Add workflow to verify linux release packages (#17904)" (#17942)
This reverts commit 3368f14fab500ebe9f6aeab5631dd1d5f5a453e5.
* Fixes Secondary ConnectCA update (#17846)
This fixes a bug that was identified which resulted in subsequent
ConnectCA configuration update not to persist in the cluster.
* fixing typo in link to jwt-validations-with-intentions doc (#17955)
* Fix streaming backend link (#17958)
* Fix streaming backend link
* Update health.mdx
* Dynamically create jwks clusters for jwt-providers (#17944)
* website: remove deprecated agent rpc docs (#17962)
* Fix missing BalanceOutboundConnections in v2 catalog. (#17964)
* feature - [NET - 4005] - [Supportability] Reloadable Configuration - enable_debug (#17565)
* # This is a combination of 9 commits.
# This is the 1st commit message:
init without tests
# This is the commit message #2:
change log
# This is the commit message #3:
fix tests
# This is the commit message #4:
fix tests
# This is the commit message #5:
added tests
# This is the commit message #6:
change log breaking change
# This is the commit message #7:
removed breaking change
# This is the commit message #8:
fix test
# This is the commit message #9:
keeping the test behaviour same
* # This is a combination of 12 commits.
# This is the 1st commit message:
init without tests
# This is the commit message #2:
change log
# This is the commit message #3:
fix tests
# This is the commit message #4:
fix tests
# This is the commit message #5:
added tests
# This is the commit message #6:
change log breaking change
# This is the commit message #7:
removed breaking change
# This is the commit message #8:
fix test
# This is the commit message #9:
keeping the test behaviour same
# This is the commit message #10:
made enable debug atomic bool
# This is the commit message #11:
fix lint
# This is the commit message #12:
fix test true enable debug
* parent 10f500e895d92cc3691ade7b74a33db755d22039
author absolutelightning <ashesh.vidyut@hashicorp.com> 1687352587 +0530
committer absolutelightning <ashesh.vidyut@hashicorp.com> 1687352592 +0530
init without tests
change log
fix tests
fix tests
added tests
change log breaking change
removed breaking change
fix test
keeping the test behaviour same
made enable debug atomic bool
fix lint
fix test true enable debug
using enable debug in agent as atomic bool
test fixes
fix tests
fix tests
added update on correct locaiton
fix tests
fix reloadable config enable debug
fix tests
fix init and acl 403
* revert commit
* Fix formatting codeblocks on APIgw docs (#17970)
* fix formatting codeblocks
* remove unnecessary indents
* Remove POC code (#17974)
* update doc (#17910)
* update doc
* update link
* Remove duplicate and unused newDecodeConfigEntry func (#17979)
* docs: samenessGroup YAML examples (#17984)
* configuration entry syntax
* Example config
* Add changelog entry for 1.16.0 (#17987)
* Fix typo (#17198)
servcies => services
* Expose JWKS cluster config through JWTProviderConfigEntry (#17978)
* Expose JWKS cluster config through JWTProviderConfigEntry
* fix typos, rename trustedCa to trustedCA
* Integration test for ext-authz Envoy extension (#17980)
* Fix incorrect protocol for transparent proxy upstreams. (#17894)
This PR fixes a bug that was introduced in:
https://github.com/hashicorp/consul/pull/16021
A user setting a protocol in proxy-defaults would cause tproxy implicit
upstreams to not honor the upstream service's protocol set in its
`ServiceDefaults.Protocol` field, and would instead always use the
proxy-defaults value.
Due to the fact that upstreams configured with "tcp" can successfully contact
upstream "http" services, this issue was not recognized until recently (a
proxy-defaults with "tcp" and a listening service with "http" would make
successful requests, but not the opposite).
As a temporary work-around, users experiencing this issue can explicitly set
the protocol on the `ServiceDefaults.UpstreamConfig.Overrides`, which should
take precedence.
The fix in this PR removes the proxy-defaults protocol from the wildcard
upstream that tproxy uses to configure implicit upstreams. When the protocol
was included, it would always overwrite the value during discovery chain
compilation, which was not correct. The discovery chain compiler also consumes
proxy defaults to determine the protocol, so simply excluding it from the
wildcard upstream config map resolves the issue.
* feat: include nodes count in operator usage endpoint and cli command (#17939)
* feat: update operator usage api endpoint to include nodes count
* feat: update operator usange cli command to includes nodes count
* [OSS] Improve Gateway Test Coverage of Catalog Health (#18011)
* fix(cli): remove failing check from 'connect envoy' registration for api gateway
* test(integration): add tests to check catalog statsus of gateways on startup
* remove extra sleep comment
* Update test/integration/consul-container/libs/assert/service.go
* changelog
* Fixes Traffic rate limitting docs (#17997)
* Fix removed service-to-service peering links (#17221)
* docs: fix removed service-to-service peering links
* docs: extend peering-via-mesh-gateways intro (thanks @trujillo-adam)
---------
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* docs: Sameness "beta" warning (#18017)
* Warning updates
* .x
* updated typo in tab heading (#18022)
* updated typo in tab heading
* updated tab group typo, too
* Document that DNS lookups can target cluster peers (#17990)
Static DNS lookups, in addition to explicitly targeting a datacenter,
can target a cluster peer. This was added in 95dc0c7b301b70a6b955a8b7c9737c9b86f03df6 but didn't make the documentation.
The driving function for the change is `parseLocality` here: 0b1299c28d/agent/dns_oss.go (L25)
The biggest change in this is to adjust the standard lookup syntax to tie
`.<datacenter>` to `.dc` as required-together, and to append in the similar `.<cluster-peer>.peer` optional argument, both to A record and SRV record lookups.
Co-authored-by: David Yu <dyu@hashicorp.com>
* Add first integration test for jwt auth with intention (#18005)
* fix stand-in text for name field (#18030)
* removed sameness conf entry from failover nav (#18033)
* docs - add service sync annotations and k8s service weight annotation (#18032)
* Docs for https://github.com/hashicorp/consul-k8s/pull/2293
* remove versions for enterprise features since they are old
---------
Co-authored-by: Tu Nguyen <im2nguyen@users.noreply.github.com>
* docs - add jobs use case for service mesh k8s (#18037)
* docs - add jobs use case for service mesh k8s
* add code blocks
* address feedback (#18045)
* Add verify server hostname to tls default (#17155)
* [CC-5718] Remove HCP token requirement during bootstrap
* Re-add error for loading HCP management token
* backport of commit f1225f5d474406073ae9f46ddd3c51e93aee3d8a
* backport of commit 5958ae0921522707652794668233741298863ade
* backport of commit 536e6f3b3b68754cb9958f83e225e6f7d1565c08
* backport of commit a99dd82929013249c8f57f1867da8a8ed4fb93d9
* backport of commit fc680e806ee6428c3a363d066f6b093fbe2fdd20
---------
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Dan Bond <danbond@protonmail.com>
Co-authored-by: Michael Zalimeni <michael.zalimeni@hashicorp.com>
Co-authored-by: Ronald <roncodingenthusiast@users.noreply.github.com>
Co-authored-by: Eric Haberkorn <erichaberkorn@gmail.com>
Co-authored-by: Andrew Stucki <andrew.stucki@hashicorp.com>
Co-authored-by: Luke Kysow <1034429+lkysow@users.noreply.github.com>
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
Co-authored-by: David Yu <dyu@hashicorp.com>
Co-authored-by: Bryce Kalow <bkalow@hashicorp.com>
Co-authored-by: Paul Glass <pglass@hashicorp.com>
Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com>
Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
Co-authored-by: Poonam Jadhav <poonam.jadhav@hashicorp.com>
Co-authored-by: Tu Nguyen <im2nguyen@users.noreply.github.com>
Co-authored-by: Chris Thain <32781396+cthain@users.noreply.github.com>
Co-authored-by: Hariram Sankaran <56744845+ramramhariram@users.noreply.github.com>
Co-authored-by: shanafarkas <105076572+shanafarkas@users.noreply.github.com>
Co-authored-by: Thomas Eckert <teckert@hashicorp.com>
Co-authored-by: Jeff Apple <79924108+Jeff-Apple@users.noreply.github.com>
Co-authored-by: Joshua Timmons <josh.timmons@hashicorp.com>
Co-authored-by: Ashesh Vidyut <134911583+absolutelightning@users.noreply.github.com>
Co-authored-by: Dan Stough <dan.stough@hashicorp.com>
Co-authored-by: Curt Bushko <cbushko@gmail.com>
Co-authored-by: Tobias Birkefeld <t@craxs.de>
Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com>
Co-authored-by: Semir Patel <semir.patel@hashicorp.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: chappie <6537530+chapmanc@users.noreply.github.com>
Co-authored-by: Derek Menteer <105233703+hashi-derek@users.noreply.github.com>
Co-authored-by: John Murret <john.murret@hashicorp.com>
Co-authored-by: Mark Campbell-Vincent <mnmvincent@gmail.com>
Co-authored-by: Daniel Upton <daniel@floppy.co>
Co-authored-by: Steven Zamborsky <97125550+stevenzamborsky@users.noreply.github.com>
Co-authored-by: George Bolo <george.bolo@gmail.com>
Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
Co-authored-by: wangxinyi7 <121973291+wangxinyi7@users.noreply.github.com>
Co-authored-by: cskh <hui.kang@hashicorp.com>
Co-authored-by: V. K <cn007b@gmail.com>
Co-authored-by: Iryna Shustava <ishustava@users.noreply.github.com>
Co-authored-by: Alex Simenduev <shamil.si@gmail.com>
Co-authored-by: Dhia Ayachi <dhia@hashicorp.com>
Co-authored-by: Dan Bond <danbond@protonmail.com>
Co-authored-by: sarahalsmiller <100602640+sarahalsmiller@users.noreply.github.com>
Co-authored-by: Gerard Nguyen <gerard@hashicorp.com>
Co-authored-by: mr-miles <miles.waller@gmail.com>
Co-authored-by: natemollica-dev <57850649+natemollica-nm@users.noreply.github.com>
Co-authored-by: John Maguire <john.maguire@hashicorp.com>
Co-authored-by: Samantha <hello@entropy.cat>
Co-authored-by: Ranjandas <thejranjan@gmail.com>
Co-authored-by: Evan Phoenix <evan@phx.io>
Co-authored-by: Michael Hofer <karras@users.noreply.github.com>
Co-authored-by: J.C. Jones <james.jc.jones@gmail.com>
Co-authored-by: Fulvio <fulviodenza823@gmail.com>
Co-authored-by: Jeremy Jacobson <jeremy.jacobson@hashicorp.com>
Co-authored-by: Jeremy Jacobson <jjacobson93@users.noreply.github.com>
This PR explicitly enables WebSocket upgrades in Envoy's UpgradeConfig for all
proxy types. (API Gateway, Ingress, and Sidecar.)
Fixes#8283
Co-authored-by: Blake Covarrubias <blake@covarrubi.as>
Bump golang.org/x/net to 0.12.0
While not necessary to directly address CVE-2023-29406 (which should be
handled by using a patched version of Go when building), an
accompanying change to HTTP/2 error handling does impact agent code.
See https://go-review.googlesource.com/c/net/+/506995 for the HTTP/2
change.
Bump this dependency across our submodules as well for the sake of
potential indirect consumers of `x/net/http`.
Manual backport of 84cbf09185ebfc59f9fcf486d1c4983ef129bf95.
Fix a bug that wrongly trims domains when there is an overlap with DC name (#17160)
* Fix a bug that wrongly trims domains when there is an overlap with DC name
Before this change, when DC name and domain/alt-domain overlap, the domain name incorrectly trimmed from the query.
Example:
Given: datacenter = dc-test, alt-domain = test.consul.
Querying for "test-node.node.dc-test.consul" will faile, because the
code was trimming "test.consul" instead of just ".consul"
This change, fixes the issue by adding dot (.) before trimming
* trimDomain: ensure domain trimmed without modyfing original domains
* update changelog
---------
Co-authored-by: Alex Simenduev <shamil.si@gmail.com>
* backport of commit 57bb6f3d729e4d76d1043efa2fa6a46137398d32
* backport of commit b2dad880653285a975795e89b0d77a6ea2fa60f1
* backport of commit 753d3c0d3f4797b6cf2d3490df996dffa8e885de
---------
Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
## Backport
This PR is auto-generated from #17754 to be assessed for backporting due
to the inclusion of the label backport/1.16.
🚨
>**Warning** automatic cherry-pick of commits failed. If the first
commit failed,
you will see a blank no-op commit below. If at least one commit
succeeded, you
will see the cherry-picked commits up to, _not including_, the commit
where
the merge conflict occurred.
The person who merged in the original PR is:
@WenInCode
This person should manually cherry-pick the original PR into a new
backport PR,
and close this one when the manual backport PR is merged in.
> merge conflict error: unable to process merge commit:
"1c757b8a2c1160ad53421b7b8bd7f74b205c4b89", automatic backport requires
rebase workflow
The below text is copied from the body of the original PR.
---
fixes#17097 Consul version of each nodes in UI nodes section
@jkirschner-hashicorp @huikang @team @Maintainers
Updated consul version in the request to register consul.
Added this as Node MetaData.
Fetching this new metadata in UI
<img width="1512" alt="Screenshot 2023-06-15 at 4 21 33 PM"
src="https://github.com/hashicorp/consul/assets/3139634/94f7cf6b-701f-4230-b9f7-d8c4342d0737">
Also made this backward compatible and tested.
Backward compatible in this context means - If consul binary with above
PR changes is deployed to one of node, and if UI is run from this node,
then the version of not only current (upgraded) node is displayed in UI
, but also of older nodes given that they are consul servers only.
For older (non-server or client) nodes the version is not added in
NodeMeta Data and hence the version will not be displayed for them.
If a old node is consul server, the version will be displayed. As the
endpoint - "v1/internal/ui/nodes?dc=dc1" was already returning version
in service meta. This is made use of in current UI changes.
<img width="1480" alt="Screenshot 2023-06-16 at 6 58 32 PM"
src="https://github.com/hashicorp/consul/assets/3139634/257942f4-fbed-437d-a492-37849d2bec4c">
---
<details>
<summary> Overview of commits </summary>
- 931fdfc7ecdc26bb7cc20b698c5e14c1b65fcc6e -
b3e2ec1ccaca3832a088ffcac54257fa6653c6c1 -
8d0e9a54907039c09330c6cd7b9e761566af6856 -
04e5d88cca37821f6667be381c16aaa5958b5c92 -
28286a2e98f8cd66ef8593c2e2893b4db6080417 -
43e50ad38207952a9c4d04d45d08b6b8f71b31fe -
0cf1b7077cdf255596254d9dc1624a269c42b94d -
27f34ce1c2973591f75b1e38a81ccbe7cee6cee3 -
2ac76d62b8cbae76b1a903021aebb9b865e29d6e -
3d618df9ef1d10dd5056c8b1ed865839c553a0e0 -
1c757b8a2c1160ad53421b7b8bd7f74b205c4b89 -
23ce82b4cee8f74dd634dbe145313e9a56c0077d -
4dc1c9b4c5aafdb8883ef977dfa9b39da138b6cb -
85a12a92528bfa267a039a9bb258170be914abf7 -
25d30a3fa980d130a30d445d26d47ef2356cb553 -
7f1d6192dce3352e92307175848b89f91e728c24 -
5174cbff84b0795d4cb36eb8980d0d5336091ac9
</details>
---------
Co-authored-by: Vijay Srinivas <vijayraghav22@gmail.com>
Co-authored-by: John Murret <john.murret@hashicorp.com>
Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com>
* backport of commit 4034bb2b3eba81ea13bf6d3a62d27094d96ffc24
* backport of commit 9c4c3c50f07d4072bb981c16cf993118fd7f6f1d
* backport of commit 7282078993aa51915afa801bdabded0f78397cb5
---------
Co-authored-by: Tom Davies <thomas.23.davies@bt.com>
* backport of commit afa1f42cc719b13074f2f286202d8f21b8000753
* backport of commit e0970025d4c2e2702af30e642b37dd5e32561756
* backport of commit 2f2aad545b1ebcae22bb481b57115a679eb539e5
* backport of commit 4a5c9c181f50343911cd30fbb0f0475e473a2c7b
---------
Co-authored-by: Ranjandas <thejranjan@gmail.com>
Co-authored-by: Chris S. Kim <kisunji92@gmail.com>
* backport of commit 93ccfe4c1195ba0ab2d12443f25d9cf29e9e4f0c
* Ensure RSA keys are at least 2048 bits in length (#17911)
* Ensure RSA keys are at least 2048 bits in length
* Add changelog
* update key length check for FIPS compliance
* Fix no new variables error and failing to return when error exists from
validating
* clean up code for better readability
* actually return value
---------
Co-authored-by: jm96441n <john.maguire@hashicorp.com>
* backport of commit d77048f1ea7136af9f627182bc79126f9472a060
* backport of commit 301de5980e7fcc8d68d16e6cdcbd5b2afea73711
* backport of commit 436bcda5ec58abbc5a65f6d244a1e1ec26beb46c
---------
Co-authored-by: Ronald Ekambi <ronekambi@gmail.com>
* backport of commit 97c779b5a2308a05fde93247209fa6e9cd3fc310
* backport of commit dd56a6800bebc54dabd7883fddc22b25ca2bdb92
---------
Co-authored-by: Michael Zalimeni <michael.zalimeni@hashicorp.com>
* backport of commit 189786c0232009b6232713e8fe98c50654e68b46
* backport of commit fceeebd3ae85db510a5cdcf0cd8288bbacf2f59e
---------
Co-authored-by: David Yu <dyu@hashicorp.com>
* backport of commit fb2f3b61004d9ef2296b51306ddbf5b6d72679ed
* backport of commit 178abb8495ba4bb35c29a835965e3f244a385865
* backport of commit 77b399877413c6e65669659deb8962c2cc5b52f6
* backport of commit a245b326ac030f7ef3292e7bbdb58e0b2850a12f
---------
Co-authored-by: Andrew Stucki <andrew.stucki@hashicorp.com>
* backport of commit 63e87434721824902d1bd8c7ef72456615020180
* backport of commit 896f11789b89062c745433a17dc37da5b56185fd
---------
Co-authored-by: Andrew Stucki <andrew.stucki@hashicorp.com>
* backport of commit dc9c08d3b8cc1eda95a05a8b041ab2a3a5248dd0
* backport of commit 1271705a5cce5fe5e9487fed2ac965ab7aac3d59
---------
Co-authored-by: Ronald Ekambi <ronekambi@gmail.com>
Co-authored-by: Ronald <roncodingenthusiast@users.noreply.github.com>
* backport of commit d77784ba51fd6a5d598ea2b87cb6e36e0fed8e72
* backport of commit f5a557dd7a5995094b3af96f1c522d49acfe795b
* backport of commit 1d782d63c437ab16e30d5bd00a6b8c3cbad08845
---------
Co-authored-by: Ronald Ekambi <ronekambi@gmail.com>
* Sujata's peering-cli branch
* Added error message for connecting to cluster
* We can export service to peer
* export handling multiple peers
* export handles multiple peers
* export now can handle multiple services
* Export after 1st cleanup
* Successful export
* Added the namespace option
* Add .changelog entry
* go mod tidy
* Stub unit tests for peering export command
* added export in peering.go
* Adding export_test
* Moved the code to services from peers and cleaned the serviceNamespace
* Added support for exporting to partitions
* Fixed partition bug
* Added unit tests for export command
* Add multi-tenancy flags
* gofmt
* Add some helpful comments
* Exclude namespace + partition flags when running OSS
* cleaned up partition stuff
* Validate required flags differently for OSS vs. ENT
* Update success output to include only the requested consumers
* cleaned up
* fixed broken test
* gofmt
* Include all flags in OSS build
* Remove example previously added to peering command
* Move stray import into correct block
* Update changelog entry to include support for exporting to a partition
* Add required-ness label to consumer-peers flag description
* Update command/services/export/export.go
Co-authored-by: Dan Stough <dan.stough@hashicorp.com>
* Add docs placeholder for new services export command
* Moved piece of code to OSS
* Break config entry init + update into separate functions
* fixed
* Vary existing service export comparison for OSS vs. ENT
* Move OSS-specific test to export_oss_test.go
* Set config entry name based on partition being exported from
* Set namespace on added services
* Adding namespace
* Remove export documentation
We will include documentation in a followup PR
* Consolidate code from export_oss into export.go
* Consolidated export_oss_test.go and export_test.go
* Add example of partition export to command synopsis
* Allow empty peers flag if partitions flag provided
* Add test coverage for -consumer-partitions flag
* Update command/services/export/export.go
Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com>
* Update command/services/export/export.go
Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com>
* Update changelog entry
* Use "cluster peers" to clear up any possible confusion
* Update test assertions
---------
Co-authored-by: 20sr20 <sujata@hashicorp.com>
Co-authored-by: Dan Stough <dan.stough@hashicorp.com>
Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com>
This commit only contains the OSS PR (datacenter query param support).
A separate enterprise PR adds support for ap and namespace query params.
Resources in Consul can exists within scopes such as datacenters, cluster
peers, admin partitions, and namespaces. You can refer to those resources from
interfaces such as the CLI, HTTP API, DNS, and configuration files.
Some scope levels have consistent naming: cluster peers are always referred to
as "peer".
Other scope levels use a short-hand in DNS lookups...
- "ns" for namespace
- "ap" for admin partition
- "dc" for datacenter
...But use long-hand in CLI commands:
- "namespace" for namespace
- "partition" for admin partition
- and "datacenter"
However, HTTP API query parameters do not follow a consistent pattern,
supporting short-hand for some scopes but long-hand for others:
- "ns" for namespace
- "partition" for admin partition
- and "dc" for datacenter.
This inconsistency is confusing, especially for users who have been exposed to
providing scope names through another interface such as CLI or DNS queries.
This commit improves UX by consistently supporting both short-hand and
long-hand forms of the namespace, partition, and datacenter scopes in HTTP API
query parameters.
* Move hcp client to subpackage hcpclient (#16800)
* [HCP Observability] New MetricsClient (#17100)
* Client configured with TLS using HCP config and retry/throttle
* Add tests and godoc for metrics client
* close body after request
* run go mod tidy
* Remove one abstraction to use the config from deps
* Address PR feedback
* remove clone
* Extract CloudConfig and mock for future PR
* Switch to hclog.FromContext
* [HCP Observability] OTELExporter (#17128)
* Client configured with TLS using HCP config and retry/throttle
* run go mod tidy
* Remove one abstraction to use the config from deps
* Address PR feedback
* Client configured with TLS using HCP config and retry/throttle
* run go mod tidy
* Create new OTELExporter which uses the MetricsClient
Add transform because the conversion is in an /internal package
* Fix lint error
* early return when there are no metrics
* Add NewOTELExporter() function
* Downgrade to metrics SDK version: v1.15.0-rc.1
* Fix imports
* fix small nits with comments and url.URL
* Fix tests by asserting actual error for context cancellation, fix parallel, and make mock more versatile
* Cleanup error handling and clarify empty metrics case
* Fix input/expected naming in otel_transform_test.go
* add comment for metric tracking
* Add a general isEmpty method
* Add clear error types
* update to latest version 1.15.0 of OTEL
* [HCP Observability] OTELSink (#17159)
* Client configured with TLS using HCP config and retry/throttle
* run go mod tidy
* Remove one abstraction to use the config from deps
* Address PR feedback
* Client configured with TLS using HCP config and retry/throttle
* run go mod tidy
* Create new OTELExporter which uses the MetricsClient
Add transform because the conversion is in an /internal package
* Fix lint error
* early return when there are no metrics
* Add NewOTELExporter() function
* Downgrade to metrics SDK version: v1.15.0-rc.1
* Fix imports
* fix small nits with comments and url.URL
* Fix tests by asserting actual error for context cancellation, fix parallel, and make mock more versatile
* Cleanup error handling and clarify empty metrics case
* Fix input/expected naming in otel_transform_test.go
* add comment for metric tracking
* Add a general isEmpty method
* Add clear error types
* update to latest version 1.15.0 of OTEL
* Client configured with TLS using HCP config and retry/throttle
* run go mod tidy
* Remove one abstraction to use the config from deps
* Address PR feedback
* Initialize OTELSink with sync.Map for all the instrument stores.
* Moved PeriodicReader init to NewOtelReader function. This allows us to use a ManualReader for tests.
* Switch to mutex instead of sync.Map to avoid type assertion
* Add gauge store
* Clarify comments
* return concrete sink type
* Fix lint errors
* Move gauge store to be within sink
* Use context.TODO,rebase and clenaup opts handling
* Rebase onto otl exporter to downgrade metrics API to v1.15.0-rc.1
* Fix imports
* Update to latest stable version by rebasing on cc-4933, fix import, remove mutex init, fix opts error messages and use logger from ctx
* Add lots of documentation to the OTELSink
* Fix gauge store comment and check ok
* Add select and ctx.Done() check to gauge callback
* use require.Equal for attributes
* Fixed import naming
* Remove float64 calls and add a NewGaugeStore method
* Change name Store to Set in gaugeStore, add concurrency tests in both OTELSink and gauge store
* Generate 100 gauge operations
* Seperate the labels into goroutines in sink test
* Generate kv store for the test case keys to avoid using uuid
* Added a race test with 300 samples for OTELSink
* Do not pass in waitgroup and use error channel instead.
* Using SHA 7dea2225a218872e86d2f580e82c089b321617b0 to avoid build failures in otel
* Fix nits
* [HCP Observability] Init OTELSink in Telemetry (#17162)
* Move hcp client to subpackage hcpclient (#16800)
* [HCP Observability] New MetricsClient (#17100)
* Client configured with TLS using HCP config and retry/throttle
* Add tests and godoc for metrics client
* close body after request
* run go mod tidy
* Remove one abstraction to use the config from deps
* Address PR feedback
* remove clone
* Extract CloudConfig and mock for future PR
* Switch to hclog.FromContext
* [HCP Observability] New MetricsClient (#17100)
* Client configured with TLS using HCP config and retry/throttle
* Add tests and godoc for metrics client
* close body after request
* run go mod tidy
* Remove one abstraction to use the config from deps
* Address PR feedback
* remove clone
* Extract CloudConfig and mock for future PR
* Switch to hclog.FromContext
* [HCP Observability] New MetricsClient (#17100)
* Client configured with TLS using HCP config and retry/throttle
* Add tests and godoc for metrics client
* close body after request
* run go mod tidy
* Remove one abstraction to use the config from deps
* Address PR feedback
* remove clone
* Extract CloudConfig and mock for future PR
* Switch to hclog.FromContext
* Client configured with TLS using HCP config and retry/throttle
* run go mod tidy
* Remove one abstraction to use the config from deps
* Address PR feedback
* Client configured with TLS using HCP config and retry/throttle
* run go mod tidy
* Create new OTELExporter which uses the MetricsClient
Add transform because the conversion is in an /internal package
* Fix lint error
* early return when there are no metrics
* Add NewOTELExporter() function
* Downgrade to metrics SDK version: v1.15.0-rc.1
* Fix imports
* fix small nits with comments and url.URL
* Fix tests by asserting actual error for context cancellation, fix parallel, and make mock more versatile
* Cleanup error handling and clarify empty metrics case
* Fix input/expected naming in otel_transform_test.go
* add comment for metric tracking
* Add a general isEmpty method
* Add clear error types
* update to latest version 1.15.0 of OTEL
* Client configured with TLS using HCP config and retry/throttle
* run go mod tidy
* Remove one abstraction to use the config from deps
* Address PR feedback
* Initialize OTELSink with sync.Map for all the instrument stores.
* Moved PeriodicReader init to NewOtelReader function. This allows us to use a ManualReader for tests.
* Switch to mutex instead of sync.Map to avoid type assertion
* Add gauge store
* Clarify comments
* return concrete sink type
* Fix lint errors
* Move gauge store to be within sink
* Use context.TODO,rebase and clenaup opts handling
* Rebase onto otl exporter to downgrade metrics API to v1.15.0-rc.1
* Fix imports
* Update to latest stable version by rebasing on cc-4933, fix import, remove mutex init, fix opts error messages and use logger from ctx
* Add lots of documentation to the OTELSink
* Fix gauge store comment and check ok
* Add select and ctx.Done() check to gauge callback
* use require.Equal for attributes
* Fixed import naming
* Remove float64 calls and add a NewGaugeStore method
* Change name Store to Set in gaugeStore, add concurrency tests in both OTELSink and gauge store
* Generate 100 gauge operations
* Seperate the labels into goroutines in sink test
* Generate kv store for the test case keys to avoid using uuid
* Added a race test with 300 samples for OTELSink
* [HCP Observability] OTELExporter (#17128)
* Client configured with TLS using HCP config and retry/throttle
* run go mod tidy
* Remove one abstraction to use the config from deps
* Address PR feedback
* Client configured with TLS using HCP config and retry/throttle
* run go mod tidy
* Create new OTELExporter which uses the MetricsClient
Add transform because the conversion is in an /internal package
* Fix lint error
* early return when there are no metrics
* Add NewOTELExporter() function
* Downgrade to metrics SDK version: v1.15.0-rc.1
* Fix imports
* fix small nits with comments and url.URL
* Fix tests by asserting actual error for context cancellation, fix parallel, and make mock more versatile
* Cleanup error handling and clarify empty metrics case
* Fix input/expected naming in otel_transform_test.go
* add comment for metric tracking
* Add a general isEmpty method
* Add clear error types
* update to latest version 1.15.0 of OTEL
* Do not pass in waitgroup and use error channel instead.
* Using SHA 7dea2225a218872e86d2f580e82c089b321617b0 to avoid build failures in otel
* Rebase onto otl exporter to downgrade metrics API to v1.15.0-rc.1
* Initialize OTELSink with sync.Map for all the instrument stores.
* Added telemetry agent to client and init sink in deps
* Fixed client
* Initalize sink in deps
* init sink in telemetry library
* Init deps before telemetry
* Use concrete telemetry.OtelSink type
* add /v1/metrics
* Avoid returning err for telemetry init
* move sink init within the IsCloudEnabled()
* Use HCPSinkOpts in deps instead
* update golden test for configuration file
* Switch to using extra sinks in the telemetry library
* keep name MetricsConfig
* fix log in verifyCCMRegistration
* Set logger in context
* pass around MetricSink in deps
* Fix imports
* Rebased onto otel sink pr
* Fix URL in test
* [HCP Observability] OTELSink (#17159)
* Client configured with TLS using HCP config and retry/throttle
* run go mod tidy
* Remove one abstraction to use the config from deps
* Address PR feedback
* Client configured with TLS using HCP config and retry/throttle
* run go mod tidy
* Create new OTELExporter which uses the MetricsClient
Add transform because the conversion is in an /internal package
* Fix lint error
* early return when there are no metrics
* Add NewOTELExporter() function
* Downgrade to metrics SDK version: v1.15.0-rc.1
* Fix imports
* fix small nits with comments and url.URL
* Fix tests by asserting actual error for context cancellation, fix parallel, and make mock more versatile
* Cleanup error handling and clarify empty metrics case
* Fix input/expected naming in otel_transform_test.go
* add comment for metric tracking
* Add a general isEmpty method
* Add clear error types
* update to latest version 1.15.0 of OTEL
* Client configured with TLS using HCP config and retry/throttle
* run go mod tidy
* Remove one abstraction to use the config from deps
* Address PR feedback
* Initialize OTELSink with sync.Map for all the instrument stores.
* Moved PeriodicReader init to NewOtelReader function. This allows us to use a ManualReader for tests.
* Switch to mutex instead of sync.Map to avoid type assertion
* Add gauge store
* Clarify comments
* return concrete sink type
* Fix lint errors
* Move gauge store to be within sink
* Use context.TODO,rebase and clenaup opts handling
* Rebase onto otl exporter to downgrade metrics API to v1.15.0-rc.1
* Fix imports
* Update to latest stable version by rebasing on cc-4933, fix import, remove mutex init, fix opts error messages and use logger from ctx
* Add lots of documentation to the OTELSink
* Fix gauge store comment and check ok
* Add select and ctx.Done() check to gauge callback
* use require.Equal for attributes
* Fixed import naming
* Remove float64 calls and add a NewGaugeStore method
* Change name Store to Set in gaugeStore, add concurrency tests in both OTELSink and gauge store
* Generate 100 gauge operations
* Seperate the labels into goroutines in sink test
* Generate kv store for the test case keys to avoid using uuid
* Added a race test with 300 samples for OTELSink
* Do not pass in waitgroup and use error channel instead.
* Using SHA 7dea2225a218872e86d2f580e82c089b321617b0 to avoid build failures in otel
* Fix nits
* pass extraSinks as function param instead
* Add default interval as package export
* remove verifyCCM func
* Add clusterID
* Fix import and add t.Parallel() for missing tests
* Kick Vercel CI
* Remove scheme from endpoint path, and fix error logging
* return metrics.MetricSink for sink method
* Update SDK
* [HCP Observability] Metrics filtering and Labels in Go Metrics sink (#17184)
* Move hcp client to subpackage hcpclient (#16800)
* [HCP Observability] New MetricsClient (#17100)
* Client configured with TLS using HCP config and retry/throttle
* Add tests and godoc for metrics client
* close body after request
* run go mod tidy
* Remove one abstraction to use the config from deps
* Address PR feedback
* remove clone
* Extract CloudConfig and mock for future PR
* Switch to hclog.FromContext
* [HCP Observability] New MetricsClient (#17100)
* Client configured with TLS using HCP config and retry/throttle
* Add tests and godoc for metrics client
* close body after request
* run go mod tidy
* Remove one abstraction to use the config from deps
* Address PR feedback
* remove clone
* Extract CloudConfig and mock for future PR
* Switch to hclog.FromContext
* [HCP Observability] New MetricsClient (#17100)
* Client configured with TLS using HCP config and retry/throttle
* Add tests and godoc for metrics client
* close body after request
* run go mod tidy
* Remove one abstraction to use the config from deps
* Address PR feedback
* remove clone
* Extract CloudConfig and mock for future PR
* Switch to hclog.FromContext
* Client configured with TLS using HCP config and retry/throttle
* run go mod tidy
* Remove one abstraction to use the config from deps
* Address PR feedback
* Client configured with TLS using HCP config and retry/throttle
* run go mod tidy
* Create new OTELExporter which uses the MetricsClient
Add transform because the conversion is in an /internal package
* Fix lint error
* early return when there are no metrics
* Add NewOTELExporter() function
* Downgrade to metrics SDK version: v1.15.0-rc.1
* Fix imports
* fix small nits with comments and url.URL
* Fix tests by asserting actual error for context cancellation, fix parallel, and make mock more versatile
* Cleanup error handling and clarify empty metrics case
* Fix input/expected naming in otel_transform_test.go
* add comment for metric tracking
* Add a general isEmpty method
* Add clear error types
* update to latest version 1.15.0 of OTEL
* Client configured with TLS using HCP config and retry/throttle
* run go mod tidy
* Remove one abstraction to use the config from deps
* Address PR feedback
* Initialize OTELSink with sync.Map for all the instrument stores.
* Moved PeriodicReader init to NewOtelReader function. This allows us to use a ManualReader for tests.
* Switch to mutex instead of sync.Map to avoid type assertion
* Add gauge store
* Clarify comments
* return concrete sink type
* Fix lint errors
* Move gauge store to be within sink
* Use context.TODO,rebase and clenaup opts handling
* Rebase onto otl exporter to downgrade metrics API to v1.15.0-rc.1
* Fix imports
* Update to latest stable version by rebasing on cc-4933, fix import, remove mutex init, fix opts error messages and use logger from ctx
* Add lots of documentation to the OTELSink
* Fix gauge store comment and check ok
* Add select and ctx.Done() check to gauge callback
* use require.Equal for attributes
* Fixed import naming
* Remove float64 calls and add a NewGaugeStore method
* Change name Store to Set in gaugeStore, add concurrency tests in both OTELSink and gauge store
* Generate 100 gauge operations
* Seperate the labels into goroutines in sink test
* Generate kv store for the test case keys to avoid using uuid
* Added a race test with 300 samples for OTELSink
* [HCP Observability] OTELExporter (#17128)
* Client configured with TLS using HCP config and retry/throttle
* run go mod tidy
* Remove one abstraction to use the config from deps
* Address PR feedback
* Client configured with TLS using HCP config and retry/throttle
* run go mod tidy
* Create new OTELExporter which uses the MetricsClient
Add transform because the conversion is in an /internal package
* Fix lint error
* early return when there are no metrics
* Add NewOTELExporter() function
* Downgrade to metrics SDK version: v1.15.0-rc.1
* Fix imports
* fix small nits with comments and url.URL
* Fix tests by asserting actual error for context cancellation, fix parallel, and make mock more versatile
* Cleanup error handling and clarify empty metrics case
* Fix input/expected naming in otel_transform_test.go
* add comment for metric tracking
* Add a general isEmpty method
* Add clear error types
* update to latest version 1.15.0 of OTEL
* Do not pass in waitgroup and use error channel instead.
* Using SHA 7dea2225a218872e86d2f580e82c089b321617b0 to avoid build failures in otel
* Rebase onto otl exporter to downgrade metrics API to v1.15.0-rc.1
* Initialize OTELSink with sync.Map for all the instrument stores.
* Added telemetry agent to client and init sink in deps
* Fixed client
* Initalize sink in deps
* init sink in telemetry library
* Init deps before telemetry
* Use concrete telemetry.OtelSink type
* add /v1/metrics
* Avoid returning err for telemetry init
* move sink init within the IsCloudEnabled()
* Use HCPSinkOpts in deps instead
* update golden test for configuration file
* Switch to using extra sinks in the telemetry library
* keep name MetricsConfig
* fix log in verifyCCMRegistration
* Set logger in context
* pass around MetricSink in deps
* Fix imports
* Rebased onto otel sink pr
* Fix URL in test
* [HCP Observability] OTELSink (#17159)
* Client configured with TLS using HCP config and retry/throttle
* run go mod tidy
* Remove one abstraction to use the config from deps
* Address PR feedback
* Client configured with TLS using HCP config and retry/throttle
* run go mod tidy
* Create new OTELExporter which uses the MetricsClient
Add transform because the conversion is in an /internal package
* Fix lint error
* early return when there are no metrics
* Add NewOTELExporter() function
* Downgrade to metrics SDK version: v1.15.0-rc.1
* Fix imports
* fix small nits with comments and url.URL
* Fix tests by asserting actual error for context cancellation, fix parallel, and make mock more versatile
* Cleanup error handling and clarify empty metrics case
* Fix input/expected naming in otel_transform_test.go
* add comment for metric tracking
* Add a general isEmpty method
* Add clear error types
* update to latest version 1.15.0 of OTEL
* Client configured with TLS using HCP config and retry/throttle
* run go mod tidy
* Remove one abstraction to use the config from deps
* Address PR feedback
* Initialize OTELSink with sync.Map for all the instrument stores.
* Moved PeriodicReader init to NewOtelReader function. This allows us to use a ManualReader for tests.
* Switch to mutex instead of sync.Map to avoid type assertion
* Add gauge store
* Clarify comments
* return concrete sink type
* Fix lint errors
* Move gauge store to be within sink
* Use context.TODO,rebase and clenaup opts handling
* Rebase onto otl exporter to downgrade metrics API to v1.15.0-rc.1
* Fix imports
* Update to latest stable version by rebasing on cc-4933, fix import, remove mutex init, fix opts error messages and use logger from ctx
* Add lots of documentation to the OTELSink
* Fix gauge store comment and check ok
* Add select and ctx.Done() check to gauge callback
* use require.Equal for attributes
* Fixed import naming
* Remove float64 calls and add a NewGaugeStore method
* Change name Store to Set in gaugeStore, add concurrency tests in both OTELSink and gauge store
* Generate 100 gauge operations
* Seperate the labels into goroutines in sink test
* Generate kv store for the test case keys to avoid using uuid
* Added a race test with 300 samples for OTELSink
* Do not pass in waitgroup and use error channel instead.
* Using SHA 7dea2225a218872e86d2f580e82c089b321617b0 to avoid build failures in otel
* Fix nits
* pass extraSinks as function param instead
* Add default interval as package export
* remove verifyCCM func
* Add clusterID
* Fix import and add t.Parallel() for missing tests
* Kick Vercel CI
* Remove scheme from endpoint path, and fix error logging
* return metrics.MetricSink for sink method
* Update SDK
* Added telemetry agent to client and init sink in deps
* Add node_id and __replica__ default labels
* add function for default labels and set x-hcp-resource-id
* Fix labels tests
* Commit suggestion for getDefaultLabels
Co-authored-by: Joshua Timmons <joshua.timmons1@gmail.com>
* Fixed server.id, and t.Parallel()
* Make defaultLabels a method on the TelemetryConfig object
* Rename FilterList to lowercase filterList
* Cleanup filter implemetation by combining regex into a single one, and making the type lowercase
* Fix append
* use regex directly for filters
* Fix x-resource-id test to use mocked value
* Fix log.Error formats
* Forgot the len(opts.Label) optimization)
* Use cfg.NodeID instead
---------
Co-authored-by: Joshua Timmons <joshua.timmons1@gmail.com>
* remove replic tag (#17484)
* [HCP Observability] Add custom metrics for OTEL sink, improve logging, upgrade modules and cleanup metrics client (#17455)
* Add custom metrics for Exporter and transform operations
* Improve deps logging
Run go mod tidy
* Upgrade SDK and OTEL
* Remove the partial success implemetation and check for HTTP status code in metrics client
* Add x-channel
* cleanup logs in deps.go based on PR feedback
* Change to debug log and lowercase
* address test operation feedback
* use GetHumanVersion on version
* Fix error wrapping
* Fix metric names
* [HCP Observability] Turn off retries for now until dynamically configurable (#17496)
* Remove retries for now until dynamic configuration is possible
* Clarify comment
* Update changelog
* improve changelog
---------
Co-authored-by: Joshua Timmons <joshua.timmons1@gmail.com>
`property-override` is an extension that allows for arbitrarily
patching Envoy resources based on resource matching filters. Patch
operations resemble a subset of the JSON Patch spec with minor
differences to facilitate patching pre-defined (protobuf) schemas.
See Envoy Extension product documentation for more details.
Co-authored-by: Eric Haberkorn <eric.haberkorn@hashicorp.com>
Co-authored-by: Kyle Havlovitz <kyle@hashicorp.com>
* perf: Remove expensive reflection from raft/mesh hot path
Replaces a reflection-based copy of a struct in the mesh topology with a
deep-copy generated implementation.
This is in the hot-path of raft FSM updates, and the reflection overhead was a
substantial part of mesh registration times (~90%). This could manifest as raft
thread saturation, and resulting instability.
Co-authored-by: Joel Brandhorst <joel.brandhorst@gmail.com>
* add changelog
---------
Co-authored-by: Joel Brandhorst <joel.brandhorst@gmail.com>
Co-authored-by: John Murret <john.murret@hashicorp.com>
Fix ACL check on health endpoint
Prior to this change, the service health API would not explicitly return an
error whenever a token with invalid permissions was given, and it would instead
return empty results. With this change, a "Permission denied" error is returned
whenever data is queried. This is done to better support the agent cache, which
performs a fetch backoff sleep whenever ACL errors are encountered. Affected
endpoints are: `/v1/health/connect/` and `/v1/health/ingress/`.
* Fix namespaced peer service updates / deletes.
This change fixes a function so that namespaced services are
correctly queried when handling updates / deletes. Prior to this
change, some peered services would not correctly be un-exported.
* Add changelog.
To avoid unintended tampering with remote downstreams via service
config, refactor BasicEnvoyExtender and RuntimeConfig to disallow
typical Envoy extensions from being applied to non-local proxies.
Continue to allow this behavior for AWS Lambda and the read-only
Validate builtin extensions.
Addresses CVE-2023-2816.
* Add ACLs Enabled field to consul agent startup status message
* Add changelog
* Update startup messages to include default ACL policy configuration
* Correct import groupings
* agent: configure server lastseen timestamp
Signed-off-by: Dan Bond <danbond@protonmail.com>
* use correct config
Signed-off-by: Dan Bond <danbond@protonmail.com>
* add comments
Signed-off-by: Dan Bond <danbond@protonmail.com>
* use default age in test golden data
Signed-off-by: Dan Bond <danbond@protonmail.com>
* add changelog
Signed-off-by: Dan Bond <danbond@protonmail.com>
* fix runtime test
Signed-off-by: Dan Bond <danbond@protonmail.com>
* agent: add server_metadata
Signed-off-by: Dan Bond <danbond@protonmail.com>
* update comments
Signed-off-by: Dan Bond <danbond@protonmail.com>
* correctly check if metadata file does not exist
Signed-off-by: Dan Bond <danbond@protonmail.com>
* follow instructions for adding new config
Signed-off-by: Dan Bond <danbond@protonmail.com>
* add comments
Signed-off-by: Dan Bond <danbond@protonmail.com>
* update comments
Signed-off-by: Dan Bond <danbond@protonmail.com>
* Update agent/agent.go
Co-authored-by: Dan Upton <daniel@floppy.co>
* agent/config: add validation for duration with min
Signed-off-by: Dan Bond <danbond@protonmail.com>
* docs: add new server_rejoin_age_max config definition
Signed-off-by: Dan Bond <danbond@protonmail.com>
* agent: add unit test for checking server last seen
Signed-off-by: Dan Bond <danbond@protonmail.com>
* agent: log continually for 60s before erroring
Signed-off-by: Dan Bond <danbond@protonmail.com>
* pr comments
Signed-off-by: Dan Bond <danbond@protonmail.com>
* remove unneeded todo
* agent: fix error message
Signed-off-by: Dan Bond <danbond@protonmail.com>
---------
Signed-off-by: Dan Bond <danbond@protonmail.com>
Co-authored-by: Dan Upton <daniel@floppy.co>
The grpc resolver implementation is fed from changes to the
router.Router. Within the router there is a map of various areas storing
the addressing information for servers in those areas. All map entries
are of the WAN variety except a single special entry for the LAN.
Addressing information in the LAN "area" are local addresses intended
for use when making a client-to-server or server-to-server request.
The client agent correctly updates this LAN area when receiving lan serf
events, so by extension the grpc resolver works fine in that scenario.
The server agent only initially populates a single entry in the LAN area
(for itself) on startup, and then never mutates that area map again.
For normal RPCs a different structure is used for LAN routing.
Additionally when selecting a server to contact in the local datacenter
it will randomly select addresses from either the LAN or WAN addressed
entries in the map.
Unfortunately this means that the grpc resolver stack as it exists on
server agents is either broken or only accidentally functions by having
servers dial each other over the WAN-accessible address. If the operator
disables the serf wan port completely likely this incidental functioning
would break.
This PR enforces that local requests for servers (both for stale reads
or leader forwarded requests) exclusively use the LAN "area" information
and also fixes it so that servers keep that area up to date in the
router.
A test for the grpc resolver logic was added, as well as a higher level
full-stack test to ensure the externally perceived bug does not return.
* snapshot: some improvments to the snapshot process
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
Fix multiple issues related to proxycfg health queries.
1. The datacenter was not being provided to a proxycfg query, which resulted in
bypassing agentless query optimizations and using the normal API instead.
2. The health rpc endpoint would return a zero index when insufficient ACLs were
detected. This would result in the agent cache performing an infinite loop of
queries in rapid succession without backoff.
Fix issue with peer stream node cleanup.
This commit encompasses a few problems that are closely related due to their
proximity in the code.
1. The peerstream utilizes node IDs in several locations to determine which
nodes / services / checks should be cleaned up or created. While VM deployments
with agents will likely always have a node ID, agentless uses synthetic nodes
and does not populate the field. This means that for consul-k8s deployments, all
services were likely bundled together into the same synthetic node in some code
paths (but not all), resulting in strange behavior. The Node.Node field should
be used instead as a unique identifier, as it should always be populated.
2. The peerstream cleanup process for unused nodes uses an incorrect query for
node deregistration. This query is NOT namespace aware and results in the node
(and corresponding services) being deregistered prematurely whenever it has zero
default-namespace services and 1+ non-default-namespace services registered on
it. This issue is tricky to find due to the incorrect logic mentioned in #1,
combined with the fact that the affected services must be co-located on the same
node as the currently deregistering service for this to be encountered.
3. The stream tracker did not understand differences between services in
different namespaces and could therefore report incorrect numbers. It was
updated to utilize the full service name to avoid conflicts and return proper
results.
* update go version to 1.20.3
* add changelog
* rename changelog file to remove underscore
* update to use 1.20.4
* update change log entry to reflect 1.20.4
When using vault as a CA and generating the local signing cert, try to
enable the PKI endpoint's auto-tidy feature with it set to tidy expired
issuers.
This adds filtering for service-defaults: consul config list -filter 'MutualTLSMode == "permissive"'.
It adds CLI warnings when the CLI writes a config entry and sees that either service-defaults or proxy-defaults contains MutualTLSMode=permissive, or sees that the mesh config entry contains AllowEnablingPermissiveMutualTLSMode=true.
Partitioned downstreams with peered upstreams could not properly merge central config info (i.e. proxy-defaults and service-defaults things like mesh gateway modes) if the upstream had an empty DestinationPartition field in Enterprise.
Due to data flow, if this setup is done using Consul client agents the field is never empty and thus does not experience the bug.
When a service is registered directly to the catalog as is the case for consul-dataplane use this field may be empty and and the internal machinery of the merging function doesn't handle this well.
This PR ensures the internal machinery of that function is referentially self-consistent.
* Persist HCP management token from server config
We want to move away from injecting an initial management token into
Consul clusters linked to HCP. The reasoning is that by using a separate
class of token we can have more flexibility in terms of allowing HCP's
token to co-exist with the user's management token.
Down the line we can also more easily adjust the permissions attached to
HCP's token to limit it's scope.
With these changes, the cloud management token is like the initial
management token in that iit has the same global management policy and
if it is created it effectively bootstraps the ACL system.
* Update SDK and mock HCP server
The HCP management token will now be sent in a special field rather than
as Consul's "initial management" token configuration.
This commit also updates the mock HCP server to more accurately reflect
the behavior of the CCM backend.
* Refactor HCP bootstrapping logic and add tests
We want to allow users to link Consul clusters that already exist to
HCP. Existing clusters need care when bootstrapped by HCP, since we do
not want to do things like change ACL/TLS settings for a running
cluster.
Additional changes:
* Deconstruct MaybeBootstrap so that it can be tested. The HCP Go SDK
requires HTTPS to fetch a token from the Auth URL, even if the backend
server is mocked. By pulling the hcp.Client creation out we can modify
its TLS configuration in tests while keeping the secure behavior in
production code.
* Add light validation for data received/loaded.
* Sanitize initial_management token from received config, since HCP will
only ever use the CloudConfig.MangementToken.
* Add changelog entry
* Move status condition for invalid certifcate to reference the listener
that is using the certificate
* Fix where we set the condition status for listeners and certificate
refs, added tests
* Add changelog
* Add MaxEjectionPercent to config entry
* Add BaseEjectionTime to config entry
* Add MaxEjectionPercent and BaseEjectionTime to protobufs
* Add MaxEjectionPercent and BaseEjectionTime to api
* Fix integration test breakage
* Verify MaxEjectionPercent and BaseEjectionTime in integration test upstream confings
* Website docs for MaxEjectionPercent and BaseEjection time
* Add `make docs` to browse docs at http://localhost:3000
* Changelog entry
* so that is the difference between consul-docker and dev-docker
* blah
* update proto funcs
* update proto
---------
Co-authored-by: Maliz <maliheh.monshizadeh@hashicorp.com>
* Bump raft to 1.5.0
* Add CHANGELOG entry
* Add CHANGELOG entry with right extension (thanks VSCode)
* Add CHANGELOG entry with right extension (thanks VSCode)
* Go mod tidy
This implements permissive mTLS , which allows toggling services into "permissive" mTLS mode.
Permissive mTLS mode allows incoming "non Consul-mTLS" traffic to be forward unmodified to the application.
* Update service-defaults and proxy-defaults config entries with a MutualTLSMode field
* Update the mesh config entry with an AllowEnablingPermissiveMutualTLS field and implement the necessary validation. AllowEnablingPermissiveMutualTLS must be true to allow changing to MutualTLSMode=permissive, but this does not require that all proxy-defaults and service-defaults are currently in strict mode.
* Update xDS listener config to add a "permissive filter chain" when MutualTLSMode=permissive for a particular service. The permissive filter chain matches incoming traffic by the destination port. If the destination port matches the service port from the catalog, then no mTLS is required and the traffic sent is forwarded unmodified to the application.
* Add a test to reproduce the race condition
* Fix race condition by publishing the event after the commit and adding a lock to prevent out of order events.
* split publish to generate the list of events before committing the transaction.
* add changelog
* remove extra func
* Apply suggestions from code review
Co-authored-by: Dan Upton <daniel@floppy.co>
* add comment to explain test
---------
Co-authored-by: Dan Upton <daniel@floppy.co>
Prior to this change, peer services would be targeted by service-default
overrides as long as the new `peer` field was not found in the config entry.
This commit removes that deprecated backwards-compatibility behavior. Now
it is necessary to specify the `peer` field in order for upstream overrides
to apply to a peer upstream.
Currently, if an acceptor peer deletes a peering the dialer's peering
will eventually get to a "terminated" state. If the two clusters need to
be re-peered the acceptor will re-generate the token but the dialer will
encounter this error on the call to establish:
"failed to get addresses to dial peer: failed to refresh peer server
addresses, will continue to use initial addresses: there is no active
peering for "<<<ID>>>""
This is because in `exchangeSecret().GetDialAddresses()` we will get an
error if fetching addresses for an inactive peering. The peering shows
up as inactive at this point because of the existing terminated state.
Rather than checking whether a peering is active we can instead check
whether it was deleted. This way users do not need to delete terminated
peerings in the dialing cluster before re-establishing them.
* Use merge of enterprise meta's rather than new custom method
* Add merge logic for tcp routes
* Add changelog
* Normalize certificate refs on gateways
* Fix infinite call loop
* Explicitly call enterprise meta
This commit swaps the partition field to the local partition for
discovery chains targeting peers. Prior to this change, peer upstreams
would always use a value of default regardless of which partition they
exist in. This caused several issues in xds / proxycfg because of id
mismatches.
Some prior fixes were made to deal with one-off id mismatches that this
PR also cleans up, since they are no longer needed.
hc-github-team-consul-core