Commit Graph

90 Commits

Author SHA1 Message Date
hc-github-team-consul-core 8936fe951a
Backport of [NET-5163] Support locality testing in consul-container into release/1.16.x (#18503)
backport of commit 61b7c0d76ff33f883dfad875d1a39bfd115b332b

Co-authored-by: Michael Zalimeni <michael.zalimeni@hashicorp.com>
2023-08-17 15:09:30 +00:00
hc-github-team-consul-core d1a52f31a2
Backport of [NET-5146] security: Update Go version to 1.20.7 and `x/net` to 0.13.0 into release/1.16.x (#18363)
backport of commit 905e371607112dc00c55cae53c907b989a651f61

Co-authored-by: Michael Zalimeni <michael.zalimeni@hashicorp.com>
2023-08-02 18:36:08 +00:00
hc-github-team-consul-core ea93c7b29c
Backport of Displays Consul version of each nodes in UI nodes section into release/1.16.x (#18113)
## Backport

This PR is auto-generated from #17754 to be assessed for backporting due
to the inclusion of the label backport/1.16.


🚨
>**Warning** automatic cherry-pick of commits failed. If the first
commit failed,
you will see a blank no-op commit below. If at least one commit
succeeded, you
will see the cherry-picked commits up to, _not including_, the commit
where
the merge conflict occurred.

The person who merged in the original PR is:
@WenInCode
This person should manually cherry-pick the original PR into a new
backport PR,
and close this one when the manual backport PR is merged in.

> merge conflict error: unable to process merge commit:
"1c757b8a2c1160ad53421b7b8bd7f74b205c4b89", automatic backport requires
rebase workflow



The below text is copied from the body of the original PR.

---

fixes #17097 Consul version of each nodes in UI nodes section

@jkirschner-hashicorp @huikang @team @Maintainers

Updated consul version in the request to register consul.
Added this as Node MetaData.
Fetching this new metadata in UI

<img width="1512" alt="Screenshot 2023-06-15 at 4 21 33 PM"
src="https://github.com/hashicorp/consul/assets/3139634/94f7cf6b-701f-4230-b9f7-d8c4342d0737">

Also made this backward compatible and tested.

Backward compatible in this context means - If consul binary with above
PR changes is deployed to one of node, and if UI is run from this node,
then the version of not only current (upgraded) node is displayed in UI
, but also of older nodes given that they are consul servers only.
For older (non-server or client) nodes the version is not added in
NodeMeta Data and hence the version will not be displayed for them.
If a old node is consul server, the version will be displayed. As the
endpoint - "v1/internal/ui/nodes?dc=dc1" was already returning version
in service meta. This is made use of in current UI changes.

<img width="1480" alt="Screenshot 2023-06-16 at 6 58 32 PM"
src="https://github.com/hashicorp/consul/assets/3139634/257942f4-fbed-437d-a492-37849d2bec4c">




---

<details>
<summary> Overview of commits </summary>

- 931fdfc7ecdc26bb7cc20b698c5e14c1b65fcc6e -
b3e2ec1ccaca3832a088ffcac54257fa6653c6c1 -
8d0e9a54907039c09330c6cd7b9e761566af6856 -
04e5d88cca37821f6667be381c16aaa5958b5c92 -
28286a2e98f8cd66ef8593c2e2893b4db6080417 -
43e50ad38207952a9c4d04d45d08b6b8f71b31fe -
0cf1b7077cdf255596254d9dc1624a269c42b94d -
27f34ce1c2973591f75b1e38a81ccbe7cee6cee3 -
2ac76d62b8cbae76b1a903021aebb9b865e29d6e -
3d618df9ef1d10dd5056c8b1ed865839c553a0e0 -
1c757b8a2c1160ad53421b7b8bd7f74b205c4b89 -
23ce82b4cee8f74dd634dbe145313e9a56c0077d -
4dc1c9b4c5aafdb8883ef977dfa9b39da138b6cb -
85a12a92528bfa267a039a9bb258170be914abf7 -
25d30a3fa980d130a30d445d26d47ef2356cb553 -
7f1d6192dce3352e92307175848b89f91e728c24 -
5174cbff84b0795d4cb36eb8980d0d5336091ac9

</details>

---------

Co-authored-by: Vijay Srinivas <vijayraghav22@gmail.com>
Co-authored-by: John Murret <john.murret@hashicorp.com>
Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com>
2023-07-17 17:27:50 +00:00
hc-github-team-consul-core dcf9b04c34
Backport of backport ent changes to oss into release/1.16.x (#17617)
* backport of commit dc9c08d3b8cc1eda95a05a8b041ab2a3a5248dd0

* backport of commit 1271705a5cce5fe5e9487fed2ac965ab7aac3d59

---------

Co-authored-by: Ronald Ekambi <ronekambi@gmail.com>
Co-authored-by: Ronald <roncodingenthusiast@users.noreply.github.com>
2023-06-08 16:45:28 +00:00
Matt Keeler 08f9cd1678
Implement the service endpoints controller (#17216)
* Add a ReplaceType dep mapper and move them into their own file

* Implement the service endpoints controller

* Implement a Catalog Controllers Integration Test
2023-06-06 17:09:48 -04:00
Matt Keeler e909289454
Various bits of cleanup detected when using Go Workspaces (#17462)
TLDR with many modules the versions included in each diverged quite a bit. Attempting to use Go Workspaces produces a bunch of errors.

This commit:

1. Fixes envoy-library-references.sh to work again
2. Ensures we are pulling in go-control-plane@v0.11.0 everywhere (previously it was at that version in some modules and others were much older)
3. Remove one usage of golang/protobuf that caused us to have a direct dependency on it.
4. Remove deprecated usage of the Endpoint field in the grpc resolver.Target struct. The current version of grpc (v1.55.0) has removed that field and recommended replacement with URL.Opaque and calls to the Endpoint() func when needing to consume the previous field.
4. `go work init <all the paths to go.mod files>` && `go work sync`. This syncrhonized versions of dependencies from the main workspace/root module to all submodules
5. Updated .gitignore to ignore the go.work and go.work.sum files. This seems to be standard practice at the moment.
6. Update doc comments in protoc-gen-consul-rate-limit to be go fmt compatible
7. Upgraded makefile infra to perform linting, testing and go mod tidy on all modules in a flexible manner.
8. Updated linter rules to prevent usage of golang/protobuf
9. Updated a leader peering test to account for an extra colon in a grpc error message.
2023-06-05 16:08:39 -04:00
cskh 762a69cc42
Test: add noCleanup to TestServer stop (#16919) 2023-04-07 20:47:54 -04:00
Ronald 71fb0a723e
Copyright headers for missing files/folders (#16708)
* copyright headers for agent folder
2023-03-28 18:48:58 -04:00
Eric Haberkorn 0351f48bfd
allow setting locality on services and nodes (#16581) 2023-03-10 09:36:15 -05:00
Nick Irvine 69890bf5df
chore: document and unit test sdk/testutil/retry (#16049) 2023-02-21 10:48:25 -08:00
Dan Stough 133271ea69
fix: revert go mod compat for sdk,api to 1.19 (#16323) 2023-02-18 14:58:39 -05:00
Dan Stough 29497be7e8
[OSS] security: update go to 1.20.1 (#16263)
* security: update go to 1.20.1
2023-02-17 15:04:12 -05:00
Dhia Ayachi 108653b9e3
add log-drop package (#15670)
* add log-drop package

* refactor to extract level

* extract metrics

* Apply suggestions from code review

Co-authored-by: Dan Upton <daniel@floppy.co>

* fix compile errors

* change to implement a log sink

* fix tests to remove sleep

* rename and add go docs

* fix expending variadic

Co-authored-by: Dan Upton <daniel@floppy.co>
2022-12-15 12:52:48 -05:00
Derek Menteer d71f071bec
Fix SDK to support older versions of Consul. (#15423)
This change was necessary, because the configuration was always
generated with a gRPC TLS port, which did not exist in Consul 1.13,
and would result in the server failing to launch with an error.

This code checks the version of Consul and conditionally adds the
gRPC TLS port, only if the version number is greater than 1.14.
2022-11-18 10:32:01 -06:00
Kyle Schochenmaier 2b1e5f69e2
removes ioutil usage everywhere which was deprecated in go1.16 (#15297)
* update go version to 1.18 for api and sdk, go mod tidy
* removes ioutil usage everywhere which was deprecated in go1.16 in favour of io and os packages. Also introduces a lint rule which forbids use of ioutil going forward.
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>
2022-11-10 10:26:01 -06:00
Chris S. Kim ae1646706f Regenerate files according to 1.19.2 formatter 2022-10-24 16:12:08 -04:00
Iryna Shustava 46fe21a204
cli/sdk: Allow redirection to a different consul dns port (#15050) 2022-10-21 13:15:32 -06:00
freddygv a21e5799f7 Use internal server certificate for peering TLS
A previous commit introduced an internally-managed server certificate
to use for peering-related purposes.

Now the peering token has been updated to match that behavior:
- The server name matches the structure of the server cert
- The CA PEMs correspond to the Connect CA

Note that if Conect is disabled, and by extension the Connect CA, we
fall back to the previous behavior of returning the manually configured
certs and local server SNI.

Several tests were updated to use the gRPC TLS port since they enable
Connect by default. This means that the peering token will embed the
Connect CA, and the dialer will expect a TLS listener.
2022-10-07 09:05:32 -06:00
Chris S. Kim ca717ced07 Track last user of a port 2022-08-08 17:30:05 -04:00
Chris S. Kim dfaf798995 Add debugging logs to freeport 2022-08-08 15:25:17 -04:00
Luke Kysow e9960dfdf3
peering: default to false (#13963)
* defaulting to false because peering will be released as beta
* Ignore peering disabled error in bundles cachetype

Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com>
Co-authored-by: freddygv <freddy@hashicorp.com>
Co-authored-by: Matt Keeler <mjkeeler7@gmail.com>
2022-08-01 15:22:36 -04:00
R.B. Boyer 5b801db24b
peering: move peer replication to the external gRPC port (#13698)
Peer replication is intended to be between separate Consul installs and
effectively should be considered "external". This PR moves the peer
stream replication bidirectional RPC endpoint to the external gRPC
server and ensures that things continue to function.
2022-07-08 12:01:13 -05:00
R.B. Boyer 9ad10318cd
add general runstep test helper instead of copying it all over the place (#13013) 2022-05-10 15:25:51 -05:00
DanStough a050aa39b9 Update go version to 1.18.1 2022-04-18 11:41:10 -04:00
R.B. Boyer ef8cc33949
raft: update to v1.3.5 (#12325)
This includes closing some leadership transfer gaps and adding snapshot
restore progress logging.
2022-02-14 13:48:52 -06:00
Daniel Nephin 1a9a656a7f sdk: add TestLogLevel for setting log level in tests
And default log level to WARN.
2022-02-03 13:42:28 -05:00
Ashwin Venkatesh fb04a28140
Add support for 'Partition' and 'RetryJoin' (#12126)
- Adding a 'Partition' and 'RetryJoin' command allows test cases where
  one would like to spin up a Consul Agent in a non-default partition to
test use-cases that are common when enabling Admin Partition on
Kubernetes.
2022-01-20 16:49:36 -05:00
Dan Upton 088ba2edaf
[OSS] Remove remaining references to master (#11827) 2022-01-20 12:47:50 +00:00
Daniel Nephin 056a52ba64 sdk/freeport: rename Port to GetOne
For better consistency with GetN
2021-11-30 17:32:41 -05:00
Daniel Nephin 11cdf7dfd9 sdk: upgrade freeport godoc
To clarify how it works, and when it is important to use.
2021-11-30 12:41:14 -05:00
Daniel Nephin 59a7fb5ca9 testing: use httptest with freeport 2021-11-29 12:01:29 -05:00
Daniel Nephin 8219e8571e sdk: add freeport functions that use t.Cleanup 2021-11-27 15:04:43 -05:00
R.B. Boyer 83bf7ab3ff
re-run gofmt on 1.17 (#11579)
This should let freshly recompiled golangci-lint binaries using Go 1.17
pass 'make lint'
2021-11-16 12:04:01 -06:00
Nitya Dhanushkodi d396faf246
command/redirect_traffic: Redirect DNS requests to Consul if -consul-dns-ip is passed in (#11480)
* command/redirect_traffic: add rules to redirect DNS to Consul. Currently uses a hack to get the consul dns service ip, and this hack only works when the service is deployed in the same namespace as consul.
* command/redirect_traffic: redirect DNS to Consul when -consul-dns-ip is passed in
* Add unit tests to Consul DNS IP table redirect rules

Co-authored-by: Ashwin Venkatesh <ashwin@hashicorp.com>
Co-authored-by: Iryna Shustava <ishustava@users.noreply.github.com>
2021-11-10 09:10:48 -05:00
FFMMM 27227c0fd2
add root_cert_ttl option for consul connect, vault ca providers (#11428)
* add root_cert_ttl option for consul connect, vault ca providers

Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Chris S. Kim <ckim@hashicorp.com>

* add changelog, pr feedback

Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>

* Update .changelog/11428.txt, more docs

Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>

* Update website/content/docs/agent/options.mdx

Co-authored-by: Kyle Havlovitz <kylehav@gmail.com>

Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>
Co-authored-by: Kyle Havlovitz <kylehav@gmail.com>
2021-11-02 11:02:10 -07:00
Freddy 27f40ccf51
Update error texts (#11022)
Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>
2021-09-14 11:08:06 -06:00
Sergey Matyukevich 19a744191f
Allow configuring graceful stop in testutil (#10566)
* Allow configuring graceful stop in testutil

Signed-off-by: Sergey Matyukevich <s.matyukevich@gmail.com>

* add a changelog

Co-authored-by: Dhia Ayachi <dhia@hashicorp.com>
2021-09-08 11:12:54 -04:00
Iryna Shustava a75a860174
cli/sdk: Allow applying redirect-traffic rules in a provided Linux namespace (#10564) 2021-07-13 10:05:48 -06:00
R.B. Boyer f408c4d387
sdk: Stop making a special /tmp/consul-test directory for testutil.TempFile and testutil.TempDir (#10494) 2021-06-24 15:46:23 -05:00
David Bariod 728165f31e
remove a race condition in sdk testutil server stop (#10342)
* remove a race condition in sdk testutil server stop

* ensure the process has actually stopped
2021-06-09 11:30:26 -04:00
Matt Keeler 58b934133d hcs-1936: Prepare for adding license auto-retrieval to auto-config in enterprise 2021-05-24 13:20:30 -04:00
Matt Keeler 82f5cb3f08 Preparation for changing where license management is done. 2021-05-24 10:19:31 -04:00
Daniel Nephin 71d6a2bf4b Fix some test flakes
- return errors in TestAgent.Start so that the retry works correctly
- remove duplicate logging, the error is returned already
- add a missing t.Helper() to retry.Run
- properly set a.Agent to nil so that subsequent retry attempts will actually try to start
2021-05-10 13:20:45 -04:00
Iryna Shustava e7dcf9acd0
Implement traffic redirection exclusion based on proxy config and user-provided values (#10134)
* Use proxy outbound port from TransparentProxyConfig if provided
* If -proxy-id is provided to the redirect-traffic command, exclude any listener ports
  from inbound traffic redirection. This includes envoy_prometheus_bind_addr,
  envoy_stats_bind_addr, and the ListenerPort from the Expose configuration.
* Allow users to provide additional inbound and outbound ports, outbound CIDRs
  and additional user IDs to be excluded from traffic redirection.
  This affects both the traffic-redirect command and the iptables SDK package.
2021-04-29 09:21:15 -07:00
Daniel Nephin 65da18a986 submatview: fix godoc and comment typos
Co-authored-by: Paul Banks <banks@banksco.de>
2021-04-28 12:27:38 -04:00
Daniel Nephin d0b2a4a9d3 sdk/retry: support ending the iteration early
I've found this feature to be very useful in https://pkg.go.dev/gotest.tools/v3/poll#WaitOn

I have encountered a few cases where I wanted that same support, so this commit adds it.
2021-04-27 19:03:17 -04:00
Daniel Nephin f169708bb0 sdk/retry: remove the need to pass args to NextOr 2021-04-27 19:03:17 -04:00
Daniel Nephin e456945466 sdk/retry: a few small debug improvements
On a few occasions I've had to read timeout stack traces for tests and
noticed that retry.Run runs the function in a goroutine. This makes
debuging a timeout more difficult because the gourinte of the retryable
function is disconnected from the stack of the actual test. It requires
searching through the entire stack trace to find the other goroutine.

By using panic instead of runtime.Goexit() we remove the need for a
separate goroutine.

Also a few other small improvements:
* add `R.Helper` so that an assertion function can be used with both
  testing.T and retry.R.
* Pass t to `Retryer.NextOr`, and call `t.Helper` in a number of places
  so that the line number reported by `t.Log` is the line in the test
  where `retry.Run` was called, instead of some line in `retry.go` that
  is not relevant to the failure.
* improve the implementation of `dedup` by removing the need to iterate
  twice. Instad track the lines and skip any duplicate when writing to
  the buffer.
2021-04-21 17:10:46 -04:00
Iryna Shustava ff2e70f4ce
cli: Add new `consul connect redirect-traffic` command for applying traffic redirection rules when Transparent Proxy is enabled. (#9910)
* Add new consul connect redirect-traffic command for applying traffic redirection rules when Transparent Proxy is enabled.
* Add new iptables package for applying traffic redirection rules with iptables.
2021-04-09 11:48:10 -07:00
John Eikenberry fe938b9849 [SDK] change all cases of *testing.T to testing.TB
Using the interface opens up the use of all methods to benchmarks as
well as tests.
2021-03-16 15:05:39 -07:00