Commit graph

19531 commits

Author SHA1 Message Date
Kyle Havlovitz 220ca06201
Add the operator usage instances command and api endpoint (#16205)
This endpoint shows total services, connect service instances and
billable service instances in the local datacenter or globally. Billable
instances = total service instances - connect services - consul server instances.
2023-02-08 12:07:21 -08:00
Andrew Stucki fd010a326c
Add additional controller implementations (#16188)
* Add additional controller implementations

* remove additional interface

* Fix comparison checks and mark unused contexts

* Switch to time.Now().UTC()

* Add a pointer helper for shadowing loop variables

* Extract anonymous functions for readability

* clean up logging

* Add Type to the Condition proto

* Update some comments and add additional space for readability

* Address PR feedback

* Fix up dirty checks and change to pointer receiver
2023-02-08 14:50:17 -05:00
Paul Banks 50c600f93b
Adding experimental support for a more efficient LogStore implementation (#16176)
* Adding experimental support for a more efficient LogStore implementation

* Adding changelog entry

* Fix go mod tidy issues
2023-02-08 16:50:22 +00:00
cskh 1c5ca0da53
feat: envoy extension - http local rate limit (#16196)
- http local rate limit
- Apply rate limit only to local_app
- unit test and integ test
2023-02-07 21:56:15 -05:00
dependabot[bot] fc92f4f75b
Bump github.com/prometheus/client_golang from 1.4.0 to 1.14.0 (#15292)
Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.4.0 to 1.14.0.
- [Release notes](https://github.com/prometheus/client_golang/releases)
- [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prometheus/client_golang/compare/v1.4.0...v1.14.0)

---
updated-dependencies:
- dependency-name: github.com/prometheus/client_golang
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com>
2023-02-07 17:32:31 -06:00
malizz 1e9d9e2493
get upstream IPs (#16197)
* get upstream IPs

* separate test data

* fix lint issue

* fix lint issue
2023-02-07 14:57:31 -08:00
John Eikenberry 6d1d82e1ad
remove redundant vault api retry logic (#16143)
remove redundant vault api retry logic

We upgraded Vault API module version to a version that has built-in
retry logic. So this code is no longer necessary.
Also add mention of re-configuring the provider in comments.
2023-02-07 20:52:22 +00:00
skpratt 63f0a138d9
revert method name change in xds server protocol for version compatibility (#16195) 2023-02-07 14:19:09 -06:00
cskh 3deda39ca9
Upgrade test: verify the agent token is working after upgrade (#16164)
1. Upgraded agent can inherit the persisted token and join the cluster
2. Agent token prior to upgrade is still valid after upgrade
3. Enable ACL in the agent configuration
2023-02-07 14:13:19 -05:00
skpratt 8e929906fa
Update token language to distinguish Accessor and Secret ID usage (#16044)
* remove legacy tokens

* remove lingering legacy token references from docs

* update language and naming for token secrets and accessor IDs

* updates all tokenID references to clarify accessorID

* remove token type references and lookup tokens by accessorID index

* remove unnecessary constants

* replace additional tokenID param names

* Add warning info for deprecated -id parameter

Co-authored-by: Paul Glass <pglass@hashicorp.com>

* Update field comment

Co-authored-by: Paul Glass <pglass@hashicorp.com>

---------

Co-authored-by: Paul Glass <pglass@hashicorp.com>
2023-02-07 12:26:30 -06:00
malizz 1777e9ec8f
add cert tests (#16192) 2023-02-07 09:58:00 -08:00
Bryce Kalow 00468d72df
docs: update redirected links (#16179) 2023-02-07 10:36:32 -06:00
Dao Thanh Tung 122d57d4d6
Add missing doc for gRPC TLS (#16161)
Signed-off-by: dttung2905 <ttdao.2015@accountancy.smu.edu.sg>
2023-02-07 03:01:28 +00:00
malizz 86b3ed6319
exclude inbound/outbound listeners from upstreams output (#16184) 2023-02-06 18:48:55 -08:00
skpratt 9ee33dff3c
revert ui changes (#16180) 2023-02-06 23:27:49 +00:00
Paul Glass 84bef23290
Rotate Circle CI SSH Key (#16178) 2023-02-06 17:09:24 -06:00
wangxinyi7 8c6fac9d97
change log level (#16128) 2023-02-06 12:58:13 -08:00
Dhia Ayachi f36f888f55
Remove empty tags 2 (#16113)
* Add support for RemoveEmptyTags in API client

* Add changelog

---------

Co-authored-by: Rémi Lapeyre <remi.lapeyre@lenstra.fr>
2023-02-06 11:12:43 -08:00
Dhia Ayachi 0f3e935228
Net 2229/rpc reduce max retries 2 (#16165)
* feat: calculate retry wait time with exponential back-off

* test: add test for getWaitTime method

* feat: enforce random jitter between min value from previous iteration and current

* extract randomStagger to simplify tests and use Milliseconds to avoid float math.

* rename variables

* add test and rename comment

---------

Co-authored-by: Poonam Jadhav <poonam.jadhav@hashicorp.com>
2023-02-06 14:07:41 -05:00
cskh 528da14a21
docs(service-resolver): clarify the default time unit in service-resolver.ConnectTimeout (#16149)
* doc: clarify the default time unit in service-resolver.ConnectTimeout

* Update website/content/docs/connect/config-entries/service-resolver.mdx

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

---------

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
2023-02-06 18:56:51 +00:00
Anita Akaeze b382aca089
NET-2087: Restart proxy sidecar during cluster upgrade (#16140) 2023-02-06 13:09:44 -05:00
Semir Patel f9f506451e
Add links in release-notes for the last five patch releases (#16109) 2023-02-06 17:46:28 +00:00
Jared Kirschner cc36dba9e7
docs: refine server TLS Vault PKI role config (#16166)
The generate_lease=true configuration is unnecessary and generates a note about performance implications in Vault logs. Remove this configuration so that the default value of generate_lease=false is used instead.
2023-02-06 12:29:36 -05:00
Luke Kysow 1178b6e48c
Document how numRetries can't be set to 0 (#16123)
* Document how numRetries can't be set to 0

Resolves https://github.com/hashicorp/consul/issues/11816 and https://github.com/hashicorp/consul/issues/8516.

* Update website/content/docs/connect/config-entries/service-router.mdx

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

---------

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
2023-02-06 17:19:44 +00:00
Nitya Dhanushkodi 77f6b20db0
refactor: remove troubleshoot module dependency on consul top level module (#16162)
Ensure nothing in the troubleshoot go module depends on consul's top level module. This is so we can import troubleshoot into consul-k8s and not import all of consul.

* turns troubleshoot into a go module [authored by @curtbushko]
* gets the envoy protos into the troubleshoot module [authored by @curtbushko]
* adds a new go module `envoyextensions` which has xdscommon and extensioncommon folders that both the xds package and the troubleshoot package can import
* adds testing and linting for the new go modules
* moves the unit tests in `troubleshoot/validateupstream` that depend on proxycfg/xds into the xds package, with a comment describing why those tests cannot be in the troubleshoot package
* fixes all the imports everywhere as a result of these changes 

Co-authored-by: Curt Bushko <cbushko@gmail.com>
2023-02-06 09:14:35 -08:00
Poonam Jadhav 59ca3b8332
feat: client RPC is retries on ErrRetryElsewhere error and forwardRequestToLeader method retries ErrRetryLater error (#16099) 2023-02-06 11:31:25 -05:00
Thomas Eckert d81f919d85
Set codegen-tools to be a dependency of deep-copy (#16124)
* Add a little message if the user runs deep-copy without it installed

* Take codegen-tools as a dependency to deep-copy
2023-02-06 16:23:46 +00:00
skpratt 9e99a30b77
Remove legacy acl policies (#15922)
* remove legacy tokens

* remove legacy acl policies

* flatten test policies to *_prefix

* address oss feedback re: phrasing and tests
2023-02-06 15:35:52 +00:00
Curt Bushko 060b7b7084
Update helm docs based on consul-k8s release/1.0.x branch (#16157)
* docs - Docs/k8s 1.0.3 helm docs

* remove openebs entry
2023-02-03 19:00:12 -08:00
Kyle Havlovitz aa08cd38ba
command: Fix logger not initializing properly in envoy command (#16148) 2023-02-03 15:12:02 -08:00
Ashwin Venkatesh a4ac598bfd
docs: use proxy health checks when enabled (#16033) 2023-02-03 14:49:09 -08:00
John Eikenberry e0346dc81a
fix goroutine leak in renew testing (#16142)
fix goroutine leak in renew testing

Test overwrote the stopWatcher() function variable for the test without
keeping and calling the original value. The original value is the
function that stops the goroutine... so it needs to be called.
2023-02-03 22:09:34 +00:00
Dan Stough 1bf3085e40
docs(service-defaults): upstream overrides for peered services (#16122)
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
2023-02-03 17:08:06 -05:00
sarahalsmiller 09400bb85b
API Gateway Controller Logic (#16058)
* Add initial API gateway controller logic

---------

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
Co-authored-by: Andrew Stucki <andrew.stucki@hashicorp.com>
Co-authored-by: Thomas Eckert <teckert@hashicorp.com>
2023-02-03 21:55:48 +00:00
Derek Menteer 5abaaead05
[OSS] Add Peer field to service-defaults upstream overrides (#15956)
* Add Peer field to service-defaults upstream overrides.

* add api changes, compat mode for service default overrides

* Fixes based on testing

---------

Co-authored-by: DanStough <dan.stough@hashicorp.com>
2023-02-03 10:51:53 -05:00
Anita Akaeze 7921a80ad2
add assertions (#16087) 2023-02-03 10:20:22 -05:00
Paul Glass 1ad327ddf5
Use agent token for service/check deregistration during anti-entropy (#16097)
Use only the agent token for deregistration during anti-entropy

The previous behavior had the agent attempt to use the "service" token
(i.e. from the `token` field in a service definition file), and if that
was not set then it would use the agent token.

The previous behavior was problematic because, if the service token had
been deleted, the deregistration request would fail. The agent would
retry the deregistration during each anti-entropy sync, and the
situation would never resolve.

The new behavior is to only/always use the agent token for service and
check deregistration during anti-entropy. This approach is:

* Simpler: No fallback logic to try different tokens
* Faster (slightly): No time spent attempting the service token
* Correct: The agent token is able to deregister services on that
  agent's node, because:
  * node:write permissions allow deregistration of services/checks on
    that node.
  * The agent token must have node:write permission, or else the agent
    is not be able to (de)register itself into the catalog

Co-authored-by: Vesa Hagström <weeezes@gmail.com>
2023-02-03 08:45:11 -06:00
Dan Upton cc02c78ce6
rate: add prometheus definitions, docs, and clearer names (#15945) 2023-02-03 12:01:57 +00:00
Nitya Dhanushkodi 6151bcfa75
refactor: move service to service validation to troubleshoot package (#16132)
This is to reduce the dependency on xds from within the troubleshoot package.
2023-02-02 22:18:10 -08:00
malizz ffd311c2b7
validate certs and get stats (#16139) 2023-02-02 14:24:18 -08:00
Nicholas Richu 28a8de3e7e
Docs: change connect to SM for mTLS page (#16082)
* Update connect-internals.mdx

Removed most references for 'Connect' given the terminology has long been deprecated in official use.

* Apply suggestions from code review

Co-authored-by: Blake Covarrubias <blake@covarrubi.as>

* Update connect-internals.mdx

Updates based on Blakes recommendations

* Update connect-internals.mdx

---------

Co-authored-by: Blake Covarrubias <blake@covarrubi.as>
Co-authored-by: Kendall Strautman <36613477+kendallstrautman@users.noreply.github.com>
2023-02-02 09:40:35 -05:00
malizz 1477cf5a82
update troubleshoot CLI (#16129) 2023-02-01 15:11:05 -08:00
Michael Wilkerson 284cf5f062
Ent merge move envoy extension proto (#16126)
* Mw/lambda envoy extension parse region (#4107)

* updated builtin extension to parse region directly from ARN
- added a unit test
- added some comments/light refactoring

* updated golden files with proper ARNs
- ARNs need to be right format now that they are being processed

* updated tests and integration tests
- removed 'region' from all EnvoyExtension arguments
- added properly formatted ARN which includes the same region found in the removed "Region" field: 'us-east-1'

* regenerated proto files
2023-02-01 11:42:32 -08:00
malizz 84e7018087
add troubleshoot cli (#16070)
* add troubleshoot cli

* fix lint issue

* fix merge conflict

* fix lint issue
2023-02-01 11:37:30 -08:00
Derek Menteer 05edbe48c8
Add unit test and update golden files. (#16115) 2023-02-01 09:51:08 -06:00
Anita Akaeze ccae7fd123
NO_JIRA: Add function to get container status before making api call (#16116) 2023-02-01 10:48:54 -05:00
Andrew Stucki 3fe148891b
APIGateway HTTPRoute scaffolding (#15859)
* Stub Config Entries for Consul Native API Gateway (#15644)

* Add empty InlineCertificate struct and protobuf

* apigateway stubs

* new files

* Stub HTTPRoute in api pkg

* checkpoint

* Stub HTTPRoute in structs pkg

* Simplify api.APIGatewayConfigEntry to be consistent w/ other entries

* Update makeConfigEntry switch, add docstring for HTTPRouteConfigEntry

* Add TCPRoute to MakeConfigEntry, return unique Kind

* proto generated files

* Stub BoundAPIGatewayConfigEntry in agent

Since this type is only written by a controller and read by xDS, it doesn't need to be defined in the `api` pkg

* Add RaftIndex to APIGatewayConfigEntry stub

* Add new config entry kinds to validation allow-list

* Add RaftIndex to other added config entry stubs

* fix panic

* Update usage metrics assertions to include new cfg entries

* Regenerate proto w/ Go 1.19

* Run buf formatter on config_entry.proto

* Add Meta and acl.EnterpriseMeta to all new ConfigEntry types

* Remove optional interface method Warnings() for now

Will restore later if we wind up needing it

* Remove unnecessary Services field from added config entry types

* Implement GetMeta(), GetEnterpriseMeta() for added config entry types

* Add meta field to proto, name consistently w/ existing config entries

* Format config_entry.proto

* Add initial implementation of CanRead + CanWrite for new config entry types

* Add unit tests for decoding of new config entry types

* Add unit tests for parsing of new config entry types

* Add unit tests for API Gateway config entry ACLs

* Return typed PermissionDeniedError on BoundAPIGateway CanWrite

* Add unit tests for added config entry ACLs

* Add BoundAPIGateway type to AllConfigEntryKinds

* Return proper kind from BoundAPIGateway

* Add docstrings for new config entry types

* Add missing config entry kinds to proto def

* Update usagemetrics_oss_test.go

* Use utility func for returning PermissionDeniedError

* Add BoundAPIGateway to proto def

Co-authored-by: Sarah Alsmiller <sarah.alsmiller@hashicorp.com>
Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>

* Add APIGateway validation

* Fix comment

* Add additional validations

* Add cert ref validation

* Add protobuf definitions

* Tabs to spaces

* Fix up field types

* Add API structs

* Move struct fields around a bit

* EventPublisher subscriptions for Consul Native API Gateway (#15757)

* Create new event topics in subscribe proto
* Add tests for PBSubscribe func
* Make configs singular, add all configs to PBToStreamSubscribeRequest
* Add snapshot methods
* Add config_entry_events tests
* Add config entry kind to topic for new configs
* Add unit tests for snapshot methods
* Start adding integration test
* Test using the new controller code
* Update agent/consul/state/config_entry_events.go
Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
* Check value of error
Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>

* Add controller stubs for API Gateway (#15837)

* update initial stub implementation

* move files, clean up mutex references

* Remove embed, use idiomatic names for constructors

* Remove stray file introduced in merge

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>

* Initial server-side and proto defs

* drop trailing whitespace

* Add APIGateway validation (#15847)

* Add APIGateway validation

* Fix comment

* Add additional validations

* Add cert ref validation

* Add protobuf definitions

* Tabs to spaces

* Fix up field types

* Add API structs

* Move struct fields around a bit

* APIGateway InlineCertificate validation (#15856)

* Add APIGateway validation

* Add additional validations

* Add protobuf definitions

* Tabs to spaces

* Add API structs

* Move struct fields around a bit

* Add validation for InlineCertificate

* Fix ACL test

* APIGateway BoundAPIGateway validation (#15858)

* Add APIGateway validation

* Fix comment

* Add additional validations

* Add cert ref validation

* Add protobuf definitions

* Tabs to spaces

* Fix up field types

* Add API structs

* Move struct fields around a bit

* Add validation for BoundAPIGateway

* drop trailing whitespace

* APIGateway TCPRoute validation (#15855)

* Add APIGateway validation

* Fix comment

* Add additional validations

* Add cert ref validation

* Add protobuf definitions

* Tabs to spaces

* Fix up field types

* Add API structs

* Move struct fields around a bit

* Add TCPRoute normalization and validation

* Address PR feedback

* Add forgotten Status

* Add some more field docs in api package

* Fix test

* Fix bad merge

* Remove duplicate helpers

* Fix up proto defs

* Fix up stray changes

* remove extra newline

---------

Co-authored-by: Thomas Eckert <teckert@hashicorp.com>
Co-authored-by: Sarah Alsmiller <sarah.alsmiller@hashicorp.com>
Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
Co-authored-by: sarahalsmiller <100602640+sarahalsmiller@users.noreply.github.com>
2023-02-01 07:59:49 -05:00
Kyle Havlovitz 3febfa2e5d
Add a flag for enabling debug logs to the connect envoy command (#15988)
* Add a flag for enabling debug logs to the `connect envoy` command

* Update website/content/commands/connect/envoy.mdx

Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>

* Add changelog note

* Add debug log note to envoy proxy doc page

* Update website/content/docs/connect/proxies/envoy.mdx

Co-authored-by: Kendall Strautman <36613477+kendallstrautman@users.noreply.github.com>

* Wording tweak in envoy bootstrap section

---------

Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
Co-authored-by: Kendall Strautman <36613477+kendallstrautman@users.noreply.github.com>
2023-01-31 13:30:20 -08:00
Derek Menteer 5572f1584d
Add Envoy extension metrics. (#16114) 2023-01-31 14:50:30 -06:00
cskh 177c466ee1
improvement: prevent filter being added twice from any enovy extension (#16112)
* improvement: prevent filter being added twice from any enovy extension

* break if error != nil

* update test
2023-01-31 16:49:45 +00:00