Commit Graph

816 Commits

Author SHA1 Message Date
Dan Stough 29497be7e8
[OSS] security: update go to 1.20.1 (#16263)
* security: update go to 1.20.1
2023-02-17 15:04:12 -05:00
Andrew Stucki b3c26ebcf7
Add stricter validation and some normalization code for API Gateway ConfigEntries (#16304)
* Add stricter validation and some normalization code for API Gateway ConfigEntries
2023-02-17 19:22:01 +00:00
Nathan Coleman 8ea5b575d9
Fix infinite recursion in inline-certificate config entry (#16276)
* Fix infinite recursion on InlineCertificateConfigEntry

GetNamespace() + GetMeta() were calling themselves. This change also simplifies by removing nil-checking to match pre-existing config entries

Co-Authored-By: Andrew Stucki <3577250+andrewstucki@users.noreply.github.com>

* Add tests for inline-certificate

* Add alias for private key field on inline-certificate

* Use valid certificate + private key for inline-certificate tests

---------

Co-authored-by: Andrew Stucki <3577250+andrewstucki@users.noreply.github.com>
2023-02-15 13:49:34 -06:00
Andrew Stucki 58af8acab9
[API Gateway] Add integration test for HTTP routes (#16236)
* [API Gateway] Add integration test for conflicted TCP listeners

* [API Gateway] Update simple test to leverage intentions and multiple listeners

* Fix broken unit test

* [API Gateway] Add integration test for HTTP routes
2023-02-13 14:18:05 -05:00
skpratt 04fff2af26
Synthesize anonymous token pre-bootstrap when needed (#16200)
* add bootstrapping detail for acl errors

* error detail improvements

* update acl bootstrapping test coverage

* update namespace errors

* update test coverage

* consolidate error message code and update changelog

* synthesize anonymous token

* Update token language to distinguish Accessor and Secret ID usage (#16044)

* remove legacy tokens

* remove lingering legacy token references from docs

* update language and naming for token secrets and accessor IDs

* updates all tokenID references to clarify accessorID

* remove token type references and lookup tokens by accessorID index

* remove unnecessary constants

* replace additional tokenID param names

* Add warning info for deprecated -id parameter

Co-authored-by: Paul Glass <pglass@hashicorp.com>

* Update field comment

Co-authored-by: Paul Glass <pglass@hashicorp.com>

---------

Co-authored-by: Paul Glass <pglass@hashicorp.com>

* revert naming change

* add testing

* revert naming change

---------

Co-authored-by: Paul Glass <pglass@hashicorp.com>
2023-02-09 20:34:02 +00:00
Andrew Stucki 8073d1d16e
Add some fixes to allow for registering via consul connect envoy -gateway api (#16219)
* Add some fixes to allow for registering via consul connect envoy -gateway api

* Fix infinite recursion

---------

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
2023-02-09 20:28:04 +00:00
skpratt 9718079a49
ACL error improvements: incomplete bootstrapping and non-existent token (#16105)
* add bootstrapping detail for acl errors

* error detail improvements

* update acl bootstrapping test coverage

* update namespace errors

* update test coverage

* add changelog

* update message for unbootstrapped error

* consolidate error message code and update changelog

* logout message change
2023-02-08 23:49:44 +00:00
Kyle Havlovitz 220ca06201
Add the `operator usage instances` command and api endpoint (#16205)
This endpoint shows total services, connect service instances and
billable service instances in the local datacenter or globally. Billable
instances = total service instances - connect services - consul server instances.
2023-02-08 12:07:21 -08:00
Andrew Stucki fd010a326c
Add additional controller implementations (#16188)
* Add additional controller implementations

* remove additional interface

* Fix comparison checks and mark unused contexts

* Switch to time.Now().UTC()

* Add a pointer helper for shadowing loop variables

* Extract anonymous functions for readability

* clean up logging

* Add Type to the Condition proto

* Update some comments and add additional space for readability

* Address PR feedback

* Fix up dirty checks and change to pointer receiver
2023-02-08 14:50:17 -05:00
cskh 1c5ca0da53
feat: envoy extension - http local rate limit (#16196)
- http local rate limit
- Apply rate limit only to local_app
- unit test and integ test
2023-02-07 21:56:15 -05:00
skpratt 8e929906fa
Update token language to distinguish Accessor and Secret ID usage (#16044)
* remove legacy tokens

* remove lingering legacy token references from docs

* update language and naming for token secrets and accessor IDs

* updates all tokenID references to clarify accessorID

* remove token type references and lookup tokens by accessorID index

* remove unnecessary constants

* replace additional tokenID param names

* Add warning info for deprecated -id parameter

Co-authored-by: Paul Glass <pglass@hashicorp.com>

* Update field comment

Co-authored-by: Paul Glass <pglass@hashicorp.com>

---------

Co-authored-by: Paul Glass <pglass@hashicorp.com>
2023-02-07 12:26:30 -06:00
Dhia Ayachi f36f888f55
Remove empty tags 2 (#16113)
* Add support for RemoveEmptyTags in API client

* Add changelog

---------

Co-authored-by: Rémi Lapeyre <remi.lapeyre@lenstra.fr>
2023-02-06 11:12:43 -08:00
skpratt 9e99a30b77
Remove legacy acl policies (#15922)
* remove legacy tokens

* remove legacy acl policies

* flatten test policies to *_prefix

* address oss feedback re: phrasing and tests
2023-02-06 15:35:52 +00:00
Derek Menteer 5abaaead05
[OSS] Add Peer field to service-defaults upstream overrides (#15956)
* Add Peer field to service-defaults upstream overrides.

* add api changes, compat mode for service default overrides

* Fixes based on testing

---------

Co-authored-by: DanStough <dan.stough@hashicorp.com>
2023-02-03 10:51:53 -05:00
skpratt 77c95779de
add missing field to oss struct (#16094) 2023-01-28 03:52:52 +00:00
skpratt 77d805ceb6
Remove legacy acl tokens (#15947)
* remove legacy tokens

* Update test comment

Co-authored-by: Paul Glass <pglass@hashicorp.com>

* fix imports

* update docs for additional CLI changes

* add test case for anonymous token

* set deprecated api fields to json ignore and fix patch errors

* update changelog to breaking-change

* fix import

* update api docs to remove legacy reference

* fix docs nav data

---------

Co-authored-by: Paul Glass <pglass@hashicorp.com>
2023-01-27 09:17:07 -06:00
Thomas Eckert f198544270
Native API Gateway Config Entries (#15897)
* Stub Config Entries for Consul Native API Gateway (#15644)
* Add empty InlineCertificate struct and protobuf
* apigateway stubs
* Stub HTTPRoute in api pkg
* Stub HTTPRoute in structs pkg
* Simplify api.APIGatewayConfigEntry to be consistent w/ other entries
* Update makeConfigEntry switch, add docstring for HTTPRouteConfigEntry
* Add TCPRoute to MakeConfigEntry, return unique Kind
* Stub BoundAPIGatewayConfigEntry in agent
* Add RaftIndex to APIGatewayConfigEntry stub
* Add new config entry kinds to validation allow-list
* Add RaftIndex to other added config entry stubs
* Update usage metrics assertions to include new cfg entries
* Add Meta and acl.EnterpriseMeta to all new ConfigEntry types
* Remove unnecessary Services field from added config entry types
* Implement GetMeta(), GetEnterpriseMeta() for added config entry types
* Add meta field to proto, name consistently w/ existing config entries
* Format config_entry.proto
* Add initial implementation of CanRead + CanWrite for new config entry types
* Add unit tests for decoding of new config entry types
* Add unit tests for parsing of new config entry types
* Add unit tests for API Gateway config entry ACLs
* Return typed PermissionDeniedError on BoundAPIGateway CanWrite
* Add unit tests for added config entry ACLs
* Add BoundAPIGateway type to AllConfigEntryKinds
* Return proper kind from BoundAPIGateway
* Add docstrings for new config entry types
* Add missing config entry kinds to proto def
* Update usagemetrics_oss_test.go
* Use utility func for returning PermissionDeniedError
* EventPublisher subscriptions for Consul Native API Gateway (#15757)
* Create new event topics in subscribe proto
* Add tests for PBSubscribe func
* Make configs singular, add all configs to PBToStreamSubscribeRequest
* Add snapshot methods
* Add config_entry_events tests
* Add config entry kind to topic for new configs
* Add unit tests for snapshot methods
* Start adding integration test
* Test using the new controller code
* Update agent/consul/state/config_entry_events.go
* Check value of error
* Add controller stubs for API Gateway (#15837)
* update initial stub implementation
* move files, clean up mutex references
* Remove embed, use idiomatic names for constructors
* Remove stray file introduced in merge
* Add APIGateway validation (#15847)
* Add APIGateway validation
* Add additional validations
* Add cert ref validation
* Add protobuf definitions
* Fix up field types
* Add API structs
* Move struct fields around a bit
* APIGateway InlineCertificate validation (#15856)
* Add APIGateway validation
* Add additional validations
* Add protobuf definitions
* Tabs to spaces
* Add API structs
* Move struct fields around a bit
* Add validation for InlineCertificate
* Fix ACL test
* APIGateway BoundAPIGateway validation (#15858)
* Add APIGateway validation
* Add additional validations
* Add cert ref validation
* Add protobuf definitions
* Fix up field types
* Add API structs
* Move struct fields around a bit
* Add validation for BoundAPIGateway
* APIGateway TCPRoute validation (#15855)
* Add APIGateway validation
* Add additional validations
* Add cert ref validation
* Add protobuf definitions
* Fix up field types
* Add API structs
* Add TCPRoute normalization and validation
* Add forgotten Status
* Add some more field docs in api package
* Fix test
* Format imports
* Rename snapshot test variable names
* Add plumbing for Native API GW Subscriptions (#16003)

Co-authored-by: Sarah Alsmiller <sarah.alsmiller@hashicorp.com>
Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
Co-authored-by: sarahalsmiller <100602640+sarahalsmiller@users.noreply.github.com>
Co-authored-by: Andrew Stucki <andrew.stucki@hashicorp.com>
2023-01-18 22:14:34 +00:00
Paul Glass 1bf1686ebc
Add new config_file_service_registration token (#15828) 2023-01-10 10:24:02 -06:00
Thomas Eckert 8e3d0da54b
Update the README for the Consul API (#15936) 2023-01-06 21:10:56 +00:00
Eric Haberkorn 01a0142d1f
Add the Lua Envoy extension (#15906) 2023-01-06 12:13:40 -05:00
Florian Apolloner cb5389cc89
Allow Operator Generated bootstrap token (#14437)
Add support to provide an initial token via the bootstrap HTTP API, similar to hashicorp/nomad#12520
2023-01-04 20:19:33 +00:00
Dan Stough 38d65efb72
[OSS] feat: access logs for listeners and listener filters (#15864)
* feat: access logs for listeners and listener filters

* changelog

* fix integration test
2022-12-22 15:18:15 -05:00
Nitya Dhanushkodi e0e4505f44
add extensions for local service to GetExtensionConfigurations (#15871)
This gets the extensions information for the local service into the snapshot and ExtensionConfigurations for a proxy. It grabs the extensions from config entries and puts them in structs.NodeService.Proxy field, which already is copied into the config snapshot.

Also:
* add EnvoyExtensions to api.AgentService so that it matches structs.NodeService
2022-12-22 10:03:33 -08:00
Nitya Dhanushkodi 8386bf19bf
extensions: refactor serverless plugin to use extensions from config entry fields (#15817)
docs: update config entry docs and the Lambda manual registration docs

Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com>
Co-authored-by: Eric <eric@haberkorn.co>
2022-12-19 12:19:37 -08:00
cskh eff52f73be
Change field to pointer so it will be parsed as nil (#15831) 2022-12-19 10:25:36 -05:00
Dan Stough b7c51a31c4
feat: add access logging API to proxy defaults (#15780) 2022-12-13 14:52:18 -05:00
cskh 3e37a449c8
feat(ingress-gateway): support outlier detection of upstream service for ingress gateway (#15614)
* feat(ingress-gateway): support outlier detection of upstream service for ingress gateway

* changelog

Co-authored-by: Eric Haberkorn <erichaberkorn@gmail.com>
2022-12-13 11:51:37 -05:00
Dao Thanh Tung 6d519c7343
Fixing CLI ACL token processing unexpected precedence (#15274)
* Fixing CLI ACL token processing unexpected precedence

* Minor flow format and add Changelog

* Fixed failed tests and improve error logging message

* Add unit test cases and minor changes from code review

* Unset env var once the test case finishes running

* remove label FINISH
2022-12-02 12:19:52 -05:00
James Oulman 71f7f2e3dc
Add support for configuring Envoys route idle_timeout (#14340)
* Add idleTimeout

Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
Co-authored-by: Dhia Ayachi <dhia@hashicorp.com>
2022-11-29 17:43:15 -05:00
Derek Menteer 8c3d314c6c
Add 1.14.1 release updates. (#15514)
Add post-release changes for 1.14.1 updates.
2022-11-21 13:35:30 -06:00
Derek Menteer 3d82afcb01
Fix issue with connect Envoy choosing incorrect TLS settings. (#15466)
This commit fixes a situation where the API TLS configuration
incorrectly influences the GRPC port TLS configuration for XDS.
2022-11-18 14:36:20 -06:00
Derek Menteer d71f071bec
Fix SDK to support older versions of Consul. (#15423)
This change was necessary, because the configuration was always
generated with a gRPC TLS port, which did not exist in Consul 1.13,
and would result in the server failing to launch with an error.

This code checks the version of Consul and conditionally adds the
gRPC TLS port, only if the version number is greater than 1.14.
2022-11-18 10:32:01 -06:00
Derek Menteer 1f21364f7c
Consul 1.14 post-release updates (#15382)
* Update changelog with 1.14 notes.

* gomod version bumps for 1.14 release.
2022-11-15 14:22:43 -06:00
Dhia Ayachi 219a3c5bd3
Leadership transfer cmd (#14132)
* add leadership transfer command

* add RPC call test (flaky)

* add missing import

* add changelog

* add command registration

* Apply suggestions from code review

Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com>

* add the possibility of providing an id to raft leadership transfer. Add few tests.

* delete old file from cherry pick

* rename changelog filename to PR #

* rename changelog and fix import

* fix failing test

* check for OperatorWrite

Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com>

* rename from leader-transfer to transfer-leader

* remove version check and add test for operator read

* move struct to operator.go

* first pass

* add code for leader transfer in the grpc backend and tests

* wire the http endpoint to the new grpc endpoint

* remove the RPC endpoint

* remove non needed struct

* fix naming

* add mog glue to API

* fix comment

* remove dead code

* fix linter error

* change package name for proto file

* remove error wrapping

* fix failing test

* add command registration

* add grpc service mock tests

* fix receiver to be pointer

* use defined values

Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com>

* reuse MockAclAuthorizer

* add documentation

* remove usage of external.TokenFromContext

* fix failing tests

* fix proto generation

* Apply suggestions from code review

Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com>

* Apply suggestions from code review

* add more context in doc for the reason

* Apply suggestions from docs code review

Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>

* regenerate proto

* fix linter errors

Co-authored-by: github-team-consul-core <github-team-consul-core@hashicorp.com>
Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com>
Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com>
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
2022-11-14 15:35:12 -05:00
Kyle Schochenmaier 2b1e5f69e2
removes ioutil usage everywhere which was deprecated in go1.16 (#15297)
* update go version to 1.18 for api and sdk, go mod tidy
* removes ioutil usage everywhere which was deprecated in go1.16 in favour of io and os packages. Also introduces a lint rule which forbids use of ioutil going forward.
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>
2022-11-10 10:26:01 -06:00
malizz b823d79fcf
update config defaults, add docs (#15302)
* update config defaults, add docs

* update grpc tls port for non-default values

* add changelog

* Update website/content/docs/upgrading/upgrade-specific.mdx

Co-authored-by: Derek Menteer <105233703+hashi-derek@users.noreply.github.com>

* Update website/content/docs/agent/config/config-files.mdx

Co-authored-by: Derek Menteer <105233703+hashi-derek@users.noreply.github.com>

* update logic for setting grpc tls port value

* move default config to default.go, update changelog

* update docs

* Fix config tests.

* Fix linter error.

* Fix ConnectCA tests.

* Cleanup markdown on upgrade notes.

Co-authored-by: Derek Menteer <105233703+hashi-derek@users.noreply.github.com>
Co-authored-by: Derek Menteer <derek.menteer@hashicorp.com>
2022-11-09 09:29:55 -08:00
Derek Menteer a8eb047ee6
Bring back parameter ServerExternalAddresses in GenerateToken endpoint (#15267)
Re-add ServerExternalAddresses parameter in GenerateToken endpoint

This reverts commit 5e156772f6a7fba5324eb6804ae4e93c091229a6
and adds extra functionality to support newer peering behaviors.
2022-11-08 14:55:18 -06:00
Eric Haberkorn 85fb3ca3b8
Update api package exported services comment. (#15285) 2022-11-08 09:26:07 -05:00
malizz 24ddeac74b
convert stream status time fields to pointers (#15252) 2022-11-03 11:51:22 -07:00
Chris S. Kim a0ac76ecf5
Allow consul debug on non-ACL consul servers (#15155) 2022-10-27 09:25:18 -04:00
Luke Kysow 1be7b5bf35
config entry: hardcode proxy-defaults name as global (#14833)
* config entry: hardcode proxy-defaults name as global

proxy-defaults can only have the name global. Because of this,
we support not even setting the name in the config file:

```
kind = "proxy-defaults"
```

Previously, writing this would result in the output:

```
Config entry written: proxy-defaults/
```

Now it will output:

```
Config entry written: proxy-defaults/global
```

This change follows what was done for the new Mesh config entry.
2022-10-25 10:55:15 -07:00
Chris S. Kim ae1646706f Regenerate files according to 1.19.2 formatter 2022-10-24 16:12:08 -04:00
Nitya Dhanushkodi 598670e376
Remove ability to specify external addresses in GenerateToken endpoint (#14930)
* Reverts "update generate token endpoint to take external addresses (#13844)"

This reverts commit f47319b7c6b6e7c7dd720a5af927ad2d33fa536d.
2022-10-19 09:31:36 -07:00
Chris S. Kim 58c041eb6e
Merge pull request #13388 from deblasis/feature/health-checks_windows_service
Feature: Health checks windows service
2022-10-17 09:26:19 -04:00
Derek Menteer bfa4adbfce Add remote peer partition and datacenter info. 2022-10-13 10:37:41 -05:00
Chris S. Kim 9d4fb0445a Include stream-related information in peering endpoints 2022-10-10 13:20:14 -06:00
freddygv a21e5799f7 Use internal server certificate for peering TLS
A previous commit introduced an internally-managed server certificate
to use for peering-related purposes.

Now the peering token has been updated to match that behavior:
- The server name matches the structure of the server cert
- The CA PEMs correspond to the Connect CA

Note that if Conect is disabled, and by extension the Connect CA, we
fall back to the previous behavior of returning the manually configured
certs and local server SNI.

Several tests were updated to use the gRPC TLS port since they enable
Connect by default. This means that the peering token will embed the
Connect CA, and the dialer will expect a TLS listener.
2022-10-07 09:05:32 -06:00
Alex Oskotsky 4d9309327f
Add the ability to retry on reset connection to service-routers (#12890) 2022-10-05 13:06:44 -04:00
cskh 2a4d420c96
fix(api): missing peer name in query option (#14835) 2022-10-05 10:04:08 -04:00
John Murret 08203ace4a
Upgrade serf to v0.10.1 and memberlist to v0.5.0 to get memberlist size metrics and broadcast queue depth metric (#14873)
* updating to serf v0.10.1 and memberlist v0.5.0 to get memberlist size metrics and memberlist broadcast queue depth metric

* update changelog

* update changelog

* correcting changelog

* adding "QueueCheckInterval" for memberlist to test

* updating integration test containers to grab latest api
2022-10-04 17:51:37 -06:00