Commit Graph

82 Commits

Author SHA1 Message Date
Eric Haberkorn 0351f48bfd
allow setting locality on services and nodes (#16581) 2023-03-10 09:36:15 -05:00
Nick Irvine 69890bf5df
chore: document and unit test sdk/testutil/retry (#16049) 2023-02-21 10:48:25 -08:00
Dan Stough 133271ea69
fix: revert go mod compat for sdk,api to 1.19 (#16323) 2023-02-18 14:58:39 -05:00
Dan Stough 29497be7e8
[OSS] security: update go to 1.20.1 (#16263)
* security: update go to 1.20.1
2023-02-17 15:04:12 -05:00
Dhia Ayachi 108653b9e3
add log-drop package (#15670)
* add log-drop package

* refactor to extract level

* extract metrics

* Apply suggestions from code review

Co-authored-by: Dan Upton <daniel@floppy.co>

* fix compile errors

* change to implement a log sink

* fix tests to remove sleep

* rename and add go docs

* fix expending variadic

Co-authored-by: Dan Upton <daniel@floppy.co>
2022-12-15 12:52:48 -05:00
Derek Menteer d71f071bec
Fix SDK to support older versions of Consul. (#15423)
This change was necessary, because the configuration was always
generated with a gRPC TLS port, which did not exist in Consul 1.13,
and would result in the server failing to launch with an error.

This code checks the version of Consul and conditionally adds the
gRPC TLS port, only if the version number is greater than 1.14.
2022-11-18 10:32:01 -06:00
Kyle Schochenmaier 2b1e5f69e2
removes ioutil usage everywhere which was deprecated in go1.16 (#15297)
* update go version to 1.18 for api and sdk, go mod tidy
* removes ioutil usage everywhere which was deprecated in go1.16 in favour of io and os packages. Also introduces a lint rule which forbids use of ioutil going forward.
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>
2022-11-10 10:26:01 -06:00
Chris S. Kim ae1646706f Regenerate files according to 1.19.2 formatter 2022-10-24 16:12:08 -04:00
Iryna Shustava 46fe21a204
cli/sdk: Allow redirection to a different consul dns port (#15050) 2022-10-21 13:15:32 -06:00
freddygv a21e5799f7 Use internal server certificate for peering TLS
A previous commit introduced an internally-managed server certificate
to use for peering-related purposes.

Now the peering token has been updated to match that behavior:
- The server name matches the structure of the server cert
- The CA PEMs correspond to the Connect CA

Note that if Conect is disabled, and by extension the Connect CA, we
fall back to the previous behavior of returning the manually configured
certs and local server SNI.

Several tests were updated to use the gRPC TLS port since they enable
Connect by default. This means that the peering token will embed the
Connect CA, and the dialer will expect a TLS listener.
2022-10-07 09:05:32 -06:00
Chris S. Kim ca717ced07 Track last user of a port 2022-08-08 17:30:05 -04:00
Chris S. Kim dfaf798995 Add debugging logs to freeport 2022-08-08 15:25:17 -04:00
Luke Kysow e9960dfdf3
peering: default to false (#13963)
* defaulting to false because peering will be released as beta
* Ignore peering disabled error in bundles cachetype

Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com>
Co-authored-by: freddygv <freddy@hashicorp.com>
Co-authored-by: Matt Keeler <mjkeeler7@gmail.com>
2022-08-01 15:22:36 -04:00
R.B. Boyer 5b801db24b
peering: move peer replication to the external gRPC port (#13698)
Peer replication is intended to be between separate Consul installs and
effectively should be considered "external". This PR moves the peer
stream replication bidirectional RPC endpoint to the external gRPC
server and ensures that things continue to function.
2022-07-08 12:01:13 -05:00
R.B. Boyer 9ad10318cd
add general runstep test helper instead of copying it all over the place (#13013) 2022-05-10 15:25:51 -05:00
DanStough a050aa39b9 Update go version to 1.18.1 2022-04-18 11:41:10 -04:00
R.B. Boyer ef8cc33949
raft: update to v1.3.5 (#12325)
This includes closing some leadership transfer gaps and adding snapshot
restore progress logging.
2022-02-14 13:48:52 -06:00
Daniel Nephin 1a9a656a7f sdk: add TestLogLevel for setting log level in tests
And default log level to WARN.
2022-02-03 13:42:28 -05:00
Ashwin Venkatesh fb04a28140
Add support for 'Partition' and 'RetryJoin' (#12126)
- Adding a 'Partition' and 'RetryJoin' command allows test cases where
  one would like to spin up a Consul Agent in a non-default partition to
test use-cases that are common when enabling Admin Partition on
Kubernetes.
2022-01-20 16:49:36 -05:00
Dan Upton 088ba2edaf
[OSS] Remove remaining references to master (#11827) 2022-01-20 12:47:50 +00:00
Daniel Nephin 056a52ba64 sdk/freeport: rename Port to GetOne
For better consistency with GetN
2021-11-30 17:32:41 -05:00
Daniel Nephin 11cdf7dfd9 sdk: upgrade freeport godoc
To clarify how it works, and when it is important to use.
2021-11-30 12:41:14 -05:00
Daniel Nephin 59a7fb5ca9 testing: use httptest with freeport 2021-11-29 12:01:29 -05:00
Daniel Nephin 8219e8571e sdk: add freeport functions that use t.Cleanup 2021-11-27 15:04:43 -05:00
R.B. Boyer 83bf7ab3ff
re-run gofmt on 1.17 (#11579)
This should let freshly recompiled golangci-lint binaries using Go 1.17
pass 'make lint'
2021-11-16 12:04:01 -06:00
Nitya Dhanushkodi d396faf246
command/redirect_traffic: Redirect DNS requests to Consul if -consul-dns-ip is passed in (#11480)
* command/redirect_traffic: add rules to redirect DNS to Consul. Currently uses a hack to get the consul dns service ip, and this hack only works when the service is deployed in the same namespace as consul.
* command/redirect_traffic: redirect DNS to Consul when -consul-dns-ip is passed in
* Add unit tests to Consul DNS IP table redirect rules

Co-authored-by: Ashwin Venkatesh <ashwin@hashicorp.com>
Co-authored-by: Iryna Shustava <ishustava@users.noreply.github.com>
2021-11-10 09:10:48 -05:00
FFMMM 27227c0fd2
add root_cert_ttl option for consul connect, vault ca providers (#11428)
* add root_cert_ttl option for consul connect, vault ca providers

Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Chris S. Kim <ckim@hashicorp.com>

* add changelog, pr feedback

Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>

* Update .changelog/11428.txt, more docs

Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>

* Update website/content/docs/agent/options.mdx

Co-authored-by: Kyle Havlovitz <kylehav@gmail.com>

Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>
Co-authored-by: Kyle Havlovitz <kylehav@gmail.com>
2021-11-02 11:02:10 -07:00
Freddy 27f40ccf51
Update error texts (#11022)
Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>
2021-09-14 11:08:06 -06:00
Sergey Matyukevich 19a744191f
Allow configuring graceful stop in testutil (#10566)
* Allow configuring graceful stop in testutil

Signed-off-by: Sergey Matyukevich <s.matyukevich@gmail.com>

* add a changelog

Co-authored-by: Dhia Ayachi <dhia@hashicorp.com>
2021-09-08 11:12:54 -04:00
Iryna Shustava a75a860174
cli/sdk: Allow applying redirect-traffic rules in a provided Linux namespace (#10564) 2021-07-13 10:05:48 -06:00
R.B. Boyer f408c4d387
sdk: Stop making a special /tmp/consul-test directory for testutil.TempFile and testutil.TempDir (#10494) 2021-06-24 15:46:23 -05:00
David Bariod 728165f31e
remove a race condition in sdk testutil server stop (#10342)
* remove a race condition in sdk testutil server stop

* ensure the process has actually stopped
2021-06-09 11:30:26 -04:00
Matt Keeler 58b934133d hcs-1936: Prepare for adding license auto-retrieval to auto-config in enterprise 2021-05-24 13:20:30 -04:00
Matt Keeler 82f5cb3f08 Preparation for changing where license management is done. 2021-05-24 10:19:31 -04:00
Daniel Nephin 71d6a2bf4b Fix some test flakes
- return errors in TestAgent.Start so that the retry works correctly
- remove duplicate logging, the error is returned already
- add a missing t.Helper() to retry.Run
- properly set a.Agent to nil so that subsequent retry attempts will actually try to start
2021-05-10 13:20:45 -04:00
Iryna Shustava e7dcf9acd0
Implement traffic redirection exclusion based on proxy config and user-provided values (#10134)
* Use proxy outbound port from TransparentProxyConfig if provided
* If -proxy-id is provided to the redirect-traffic command, exclude any listener ports
  from inbound traffic redirection. This includes envoy_prometheus_bind_addr,
  envoy_stats_bind_addr, and the ListenerPort from the Expose configuration.
* Allow users to provide additional inbound and outbound ports, outbound CIDRs
  and additional user IDs to be excluded from traffic redirection.
  This affects both the traffic-redirect command and the iptables SDK package.
2021-04-29 09:21:15 -07:00
Daniel Nephin 65da18a986 submatview: fix godoc and comment typos
Co-authored-by: Paul Banks <banks@banksco.de>
2021-04-28 12:27:38 -04:00
Daniel Nephin d0b2a4a9d3 sdk/retry: support ending the iteration early
I've found this feature to be very useful in https://pkg.go.dev/gotest.tools/v3/poll#WaitOn

I have encountered a few cases where I wanted that same support, so this commit adds it.
2021-04-27 19:03:17 -04:00
Daniel Nephin f169708bb0 sdk/retry: remove the need to pass args to NextOr 2021-04-27 19:03:17 -04:00
Daniel Nephin e456945466 sdk/retry: a few small debug improvements
On a few occasions I've had to read timeout stack traces for tests and
noticed that retry.Run runs the function in a goroutine. This makes
debuging a timeout more difficult because the gourinte of the retryable
function is disconnected from the stack of the actual test. It requires
searching through the entire stack trace to find the other goroutine.

By using panic instead of runtime.Goexit() we remove the need for a
separate goroutine.

Also a few other small improvements:
* add `R.Helper` so that an assertion function can be used with both
  testing.T and retry.R.
* Pass t to `Retryer.NextOr`, and call `t.Helper` in a number of places
  so that the line number reported by `t.Log` is the line in the test
  where `retry.Run` was called, instead of some line in `retry.go` that
  is not relevant to the failure.
* improve the implementation of `dedup` by removing the need to iterate
  twice. Instad track the lines and skip any duplicate when writing to
  the buffer.
2021-04-21 17:10:46 -04:00
Iryna Shustava ff2e70f4ce
cli: Add new `consul connect redirect-traffic` command for applying traffic redirection rules when Transparent Proxy is enabled. (#9910)
* Add new consul connect redirect-traffic command for applying traffic redirection rules when Transparent Proxy is enabled.
* Add new iptables package for applying traffic redirection rules with iptables.
2021-04-09 11:48:10 -07:00
John Eikenberry fe938b9849 [SDK] change all cases of *testing.T to testing.TB
Using the interface opens up the use of all methods to benchmarks as
well as tests.
2021-03-16 15:05:39 -07:00
Shengjing Zhu 403e1e2534 sdk: read procfs instead of using sysctl command
This drops the dependency of sysctl command
2020-10-25 00:37:04 +08:00
R.B. Boyer d6dce2332a
connect: intentions are now managed as a new config entry kind "service-intentions" (#8834)
- Upgrade the ConfigEntry.ListAll RPC to be kind-aware so that older
copies of consul will not see new config entries it doesn't understand
replicate down.

- Add shim conversion code so that the old API/CLI method of interacting
with intentions will continue to work so long as none of these are
edited via config entry endpoints. Almost all of the read-only APIs will
continue to function indefinitely.

- Add new APIs that operate on individual intentions without IDs so that
the UI doesn't need to implement CAS operations.

- Add a new serf feature flag indicating support for
intentions-as-config-entries.

- The old line-item intentions way of interacting with the state store
will transparently flip between the legacy memdb table and the config
entry representations so that readers will never see a hiccup during
migration where the results are incomplete. It uses a piece of system
metadata to control the flip.

- The primary datacenter will begin migrating intentions into config
entries on startup once all servers in the datacenter are on a version
of Consul with the intentions-as-config-entries feature flag. When it is
complete the old state store representations will be cleared. We also
record a piece of system metadata indicating this has occurred. We use
this metadata to skip ALL of this code the next time the leader starts
up.

- The secondary datacenters continue to run the old intentions
replicator until all servers in the secondary DC and primary DC support
intentions-as-config-entries (via serf flag). Once this condition it met
the old intentions replicator ceases.

- The secondary datacenters replicate the new config entries as they are
migrated in the primary. When they detect that the primary has zeroed
it's old state store table it waits until all config entries up to that
point are replicated and then zeroes its own copy of the old state store
table. We also record a piece of system metadata indicating this has
occurred. We use this metadata to skip ALL of this code the next time
the leader starts up.
2020-10-06 13:24:05 -05:00
Paul Burlumi c28db3960e testutil: use TestingTB instead of testing.TB 2020-09-10 21:37:02 +01:00
R.B. Boyer a14a31ccf1
sdk: also print test agent logs in verbose mode (#8616) 2020-09-04 11:24:11 -05:00
Daniel Nephin b204e342c5
Merge pull request #8514 from hashicorp/dnephin/testing-improvements-1
testing: small improvements to TestSessionCreate and testutil.retry
2020-08-18 18:26:05 -04:00
Daniel Nephin 3e0d63a6b7 testing: use t.Cleanup in testutil.TempFile
So that it has the same behaviour as TempDir.

Also remove the now unnecessary 'defer os.Remove'
2020-08-14 20:06:01 -04:00
Daniel Nephin 5d53e7dbde Use t.Helper in testutil/retry 2020-08-14 18:55:52 -04:00
Daniel Nephin 2725513eea testutil: Add t.Cleanup to TempDir
TempDir registers a Cleanup so that the directory is always removed. To disable to cleanup, set the TEST_NOCLEANUP
env var.
2020-08-14 13:19:10 -04:00