Commit graph

22799 commits

Author SHA1 Message Date
James Rasell 042bf0fa57
client: hookup service wrapper for use within client hooks. 2022-03-21 10:29:57 +01:00
James Rasell fa67b0d53a
client: modify service wrapper to accomodate restore behaviour. 2022-03-21 09:49:39 +01:00
James Rasell 7718b9bc3f
Merge pull request #12298 from hashicorp/f-gh-266-nomad
client: add Nomad service registration implementation.
2022-03-21 08:58:45 +01:00
Seth Hoenig 3303a4534a
Merge pull request #12322 from hashicorp/ci-gha
ci: turn on testing in github actions
2022-03-18 12:58:09 -05:00
Seth Hoenig 8eea6e3aa3 ci: scope to push, ignore more dirs, update go update script 2022-03-18 12:47:38 -05:00
Seth Hoenig 57bd480062 ci: turn on testing in github actions 2022-03-18 11:12:24 -05:00
Seth Hoenig f2914ea36c
Merge pull request #12321 from hashicorp/ci-less-logging
ci: limit gotestsum to circle ci
2022-03-18 10:02:13 -05:00
Seth Hoenig 4d86f5d94d ci: limit gotestsum to circle ci
Part 2 of breaking up https://github.com/hashicorp/nomad/pull/12255

This PR makes it so gotestsum is invoked only in CircleCI. Also the
HCLogger(t) is plumbed more correctly in TestServer and TestAgent so
that they respect NOMAD_TEST_LOG_LEVEL.

The reason for these is we'll want to disable logging in GHA,
where spamming the disk with logs really drags performance.
2022-03-18 09:15:01 -05:00
Tim Gross 1561f66d99
api: fix ENT-only test imports for moved testutil package (#12320)
The `api/testutil` package was moved to `api/internal/testutil` but
this wasn't caught in the ENT tests because they're not run here in
the OSS repo.
2022-03-18 10:12:28 -04:00
Tim Gross 9f05d62338
E2E with HCP Consul/Vault (#12267)
Use HCP Consul and HCP Vault for the Consul and Vault clusters used in E2E testing. This has the following benefits:

* Without the need to support mTLS bootstrapping for Consul and Vault, we can simplify the mTLS configuration by leaning on Terraform instead of janky bash shell scripting.
* Vault bootstrapping is no longer required, so we can eliminate even more janky shell scripting
* Our E2E exercises HCP, which is important to us as an organization
* With the reduction in configurability, we can simplify the Terraform configuration and drop the complicated `provision.sh`/`provision.ps1` scripts we were using previously. We can template Nomad configuration files and upload them with the `file` provisioner.
* Packer builds for Linux and Windows become much simpler.

tl;dr way less janky shell scripting!
2022-03-18 09:27:28 -04:00
Seth Hoenig ab9a639a0a
Merge pull request #12313 from hashicorp/purge-parallel-2
ci: more parallel removal
2022-03-17 13:48:37 -05:00
Seth Hoenig b73d911f05 ci: do not exclude Parallel semgrep rule 2022-03-17 13:45:56 -05:00
Luiz Aoqui 68e5b58007
cli: display Raft version in server members (#12317)
The previous output of the `nomad server members` command would output a
column named `Protocol` that displayed the Serf protocol being currently
used by servers.

This is not a configurable option, so it holds very little value to
operators. It is also easy to confuse it with the Raft Protocol version,
which is configurable and highly relevant to operators.

This commit replaces the previous `Protocol` column with the new `Raft
Version`. It also updates the `-detailed` flag to be called `-verbose`
so it matches other commands. The detailed output now also outputs the
same information as the standard output with the addition of the
previous `Protocol` column and `Tags`.
2022-03-17 14:15:10 -04:00
Luiz Aoqui 15089f055f
api: add related evals to eval details (#12305)
The `related` query param is used to indicate that the request should
return a list of related (next, previous, and blocked) evaluations.

Co-authored-by: Jasmine Dahilig <jasmine@hashicorp.com>
2022-03-17 13:56:14 -04:00
Luiz Aoqui 8db12c2a17
server: transfer leadership in case of error (#12293)
When a Nomad server becomes the Raft leader, it must perform several
actions defined in the establishLeadership function. If any of these
actions fail, Raft will think the node is the leader, but it will not
actually be able to act as a Nomad leader.

In this scenario, leadership must be revoked and transferred to another
server if possible, or the node should retry the establishLeadership
steps.
2022-03-17 11:10:57 -04:00
Seth Hoenig 373d8f7241 ci: missing import for nomad09upgrade 2022-03-17 08:49:15 -05:00
Seth Hoenig 58b3d1711b ci: semgrep rule for parallel tests
Adds a semgrep rule warning about using ci.Parallel instead of t.Parallel
2022-03-17 08:43:37 -05:00
Seth Hoenig f87eb666c7 e2e: have e2e use ci.Parallel
This is a followup to having tests run in serial in CI.

The e2e package isn't in CI, but lets use the helper anyway
so we can setup semgrep rules covering the entire repository.
2022-03-17 08:37:34 -05:00
Seth Hoenig 3943dd1e16 ci: use serial testing for api in CI
This is a followup to running tests in serial in CI.
Since the API package cannot import anything outside of api/,
copy the ci.Parallel function into api/internal/testutil, and
have api tests use that.
2022-03-17 08:35:01 -05:00
James Rasell 1a4db3523d
Merge branch 'main' into tlefebvre/fix-wrong-drivernetworkmanager-interface 2022-03-17 09:38:13 +01:00
James Rasell ae26f79357
client: add Nomad service registration implementation. 2022-03-17 09:17:13 +01:00
James Rasell 54eed7b8ee
Merge pull request #12295 from hashicorp/f-gh-266-wrapper
client: add service registration wrapper to handle providers.
2022-03-17 08:52:45 +01:00
James Rasell 91e4d20b1d
Merge pull request #12307 from hashicorp/b-groupservices-avoid-double-tg-lookup
client: avoid double group lookup within groupservice hook setup.
2022-03-16 17:00:01 +01:00
Luiz Aoqui 83d834d84c
tests: move state store namespace tests from ENT (#12308) 2022-03-16 11:56:11 -04:00
Seth Hoenig aca50349f4
Merge pull request #12299 from hashicorp/ci-parallel
ci: trade test parallelization for unconstrained gomaxprocs
2022-03-16 08:55:39 -05:00
Seth Hoenig 2b83614a26 ci: explain why ci runs tests in serial now 2022-03-16 08:38:42 -05:00
James Rasell e7d0dfbc8b
client: avoid double group lookup within groupservice hook setup. 2022-03-16 09:42:57 +01:00
Thomas Lefebvre c7fbf1089c fix: update incorrect DriverNetworkManager interface implementation in plugins/drivers/client.go and drivers/mock/driver.go
And add assertions to catch drifts at compilation time.
2022-03-15 11:51:01 -07:00
Seth Hoenig 2631659551 ci: swap ci parallelization for unconstrained gomaxprocs 2022-03-15 12:58:52 -05:00
Luiz Aoqui 8cf599c7fc
fix alloc list test (#12297)
The alloc list test with pagination was creating allocs before the
target namespace existed. This works in OSS but fails in ENT because
quotas are checked before the alloc can be created, so the namespace
must exist beforehand.
2022-03-15 10:41:07 -04:00
James Rasell a44ffcee1f
client: add service registration wrapper to handle providers.
The service registration wrapper handles sending requests to
backend providers without the caller needing to know this
information. This will be used within the task and alloc runner
service hooks when performing service registration activities.
2022-03-15 12:43:52 +01:00
James Rasell d09afae9cc
Merge pull request #12272 from hashicorp/f-gh-266-refactor
client: refactor common service registration objects from Consul.
2022-03-15 10:26:33 +01:00
James Rasell 7cd28a6fb6
client: refactor common service registration objects from Consul.
This commit performs refactoring to pull out common service
registration objects into a new `client/serviceregistration`
package. This new package will form the base point for all
client specific service registration functionality.

The Consul specific implementation is not moved as it also
includes non-service registration implementations; this reduces
the blast radius of the changes as well.
2022-03-15 09:38:30 +01:00
Tim Gross 3bf948dc00
docs: clarify restart inheritance and add examples (#12275)
Clarify the behavior of `restart` inheritance with respect to Connect
sidecar tasks. Remove incorrect language about the scheduler being
involved in restart decisions. Try to make the `delay` mode
documentation more clear, and provide examples of delay vs fail.
2022-03-14 15:49:08 -04:00
Luiz Aoqui 9b393d0535
docs: initial docs for the new API features (#12094) 2022-03-14 10:58:42 -04:00
Lars Lehtonen 93cc3392ad
scheduler: fix unused dstate variable (#12268) 2022-03-14 10:00:59 -04:00
James Rasell 018a3e9f25
Merge pull request #12201 from hashicorp/f-gh-265
service discovery: add config boolean parameter and fingerprinting
2022-03-14 13:28:25 +01:00
James Rasell 1c92ca213f
Merge pull request #12190 from hashicorp/f-gh-261
config: add native service discovery admin boolean parameter.
2022-03-14 13:19:11 +01:00
James Rasell c00976acc7
Merge pull request #12189 from hashicorp/f-gh-264
jobspec: add service block provider parameter
2022-03-14 13:18:51 +01:00
James Rasell dc1378d6eb
job: add native service discovery job constraint mutator. 2022-03-14 12:42:12 +01:00
James Rasell 6d3589e8a7
client: add service discovery feature enabled attribute. 2022-03-14 12:42:01 +01:00
James Rasell 541a0f7d9a
config: add native service discovery admin boolean parameter. 2022-03-14 11:48:13 +01:00
James Rasell d18f861530
api: add service block provider parameter. 2022-03-14 10:01:20 +01:00
James Rasell 4a334c1721
hcl1: add service block provider parameter. 2022-03-14 10:00:53 +01:00
James Rasell 783d7fdc31
jobspec: add service block provider parameter and validation. 2022-03-14 09:21:20 +01:00
Luiz Aoqui 2876739a51
api: apply consistent behaviour of the reverse query parameter (#12244) 2022-03-11 19:44:52 -05:00
Luiz Aoqui a42e64c039
docs: add namespace param to job parse API (#12258) 2022-03-10 16:35:07 -05:00
Seth Hoenig 098767ce57
Merge pull request #12253 from hashicorp/hack-tiny-chroot
testing: use a smaller chroot when running exec driver tests
2022-03-09 17:12:13 -06:00
Seth Hoenig 6f0998fcad testing: use a smaller chroot when running exec driver tests
The default chroot copies all of /bin, /usr, etc. which can ammount
to gigabytes of stuff not actually needed for running our tests.

Use a smaller chroot in test cases so that CI infra with poor disk
IO has a chance.
2022-03-09 16:24:07 -06:00
Seth Hoenig abf46fa6aa
Merge pull request #12251 from hashicorp/docs-contributing-cgo
docs: describe the cgo dependency
2022-03-09 13:12:48 -06:00