Commit Graph

19693 Commits

Author SHA1 Message Date
Pierre Cauchois 1efe05f516 Do not double-remove checks removed by Consul
When deregistering a service, consul also deregisters the associated
checks. The current state keeps track of all services and all checks
separately and deregisters them in sequence, which leads, whether during
syncs or shutdowns, to check deregistrations happening twice and failing
the second time (generating errors in logs)

This fix includes:
- a fix to the sync logic that just pulls the checks *after* the
services have been synced
- a fix to the shutdown mechanism that gets an updated list of checks
after deregistering the services, so that we get a cleaner check
deregistration process.
2020-10-06 00:30:29 +00:00
Seth Hoenig 0c5ae5769f
Merge pull request #9029 from hashicorp/b-tgs-updates
consul/connect: trigger update as necessary on connect changes
2020-10-05 16:48:04 -05:00
Chris Baker 8a50ba9399
Merge pull request #9020 from hashicorp/b-8796-i-do-not-want-no-null-chars
job, task group, and task IDs should not allow null characters
2020-10-05 14:54:02 -05:00
Seth Hoenig f44a4f68ee consul/connect: trigger update as necessary on connect changes
This PR fixes a long standing bug where submitting jobs with changes
to connect services would not trigger updates as expected. Previously,
service blocks were not considered as sources of destructive updates
since they could be synced with consul non-destructively. With Connect,
task group services that have changes to their connect block or to
the service port should be destructive, since the network plumbing of
the alloc is going to need updating.

Fixes #8596 #7991

Non-destructive half in #7192
2020-10-05 14:53:00 -05:00
Chris Baker 7f701fddd0 updated docs and validation to further prohibit null chars in region, datacenter, and job name 2020-10-05 18:01:50 +00:00
Chris Baker 5c68f53c24 updated api tests wrt backwards compat on null chars in IDs 2020-10-05 18:01:50 +00:00
Chris Baker 23ea7cd27c updated job validate to refute job/group/task IDs containing null characters
updated CHANGELOG and upgrade guide
2020-10-05 18:01:49 +00:00
Chris Baker c8fd9428d4 documenting tests around null characters in job id, task group name, and task name 2020-10-05 18:01:49 +00:00
Seth Hoenig e48605809b
Merge pull request #9008 from luhhujbb/doc-portmap
[doc] fix deprecated port_map example
2020-10-05 12:25:53 -05:00
Tim Gross 12f3249390
changelog entry for Consul segment fingerprint (#9015) 2020-10-05 12:35:34 -04:00
Tim Gross 474c18102d
e2e: extend ConsulTemplate test and fix flakiness (#8997)
Add service discovery integration to the existing consul-template E2E test,
and verify both service and key updates force re-rendering. Fixes flakiness by
using the longer default wait config we use elsewhere.

Removes our last direct dependency on gomega.
2020-10-05 10:51:55 -04:00
Ryan Oaks da3e4a9e71
Merge pull request #8991 from hashicorp/ro.docs-formatting
docs: Format docs website code and mdx
2020-10-05 10:31:40 -04:00
James Rasell 2ed78b8a7e
cli: move tests to use NewMockUi func. 2020-10-05 16:07:41 +02:00
Tim Gross 727277793b
e2e: bootstrap vault and provision Nomad with vault tokens (#9010)
Provisions vault with the policies described in the Nomad Vault integration
guide, and drops a configuration file for Nomad vault server configuration
with its token. The vault root token is exposed to the E2E runner so that
tests can write additional policies to vault.
2020-10-05 09:28:37 -04:00
James Rasell b8727997cd
cli: add policy list and info to new scaling cmd.
This adds the ability to detail scaling policies via the CLI.
2020-10-05 15:18:30 +02:00
Kent 'picat' Gruber 5e1c716835
Merge pull request #8998 from hashicorp/keygen-32-bytes
Use 32-byte key for gossip encryption to enable AES-256
2020-10-02 17:17:55 -04:00
Landan Cheruka 023a2d36b7
fingerprint: changed unique.platform.azure.hostname to unique.platform.azure.name (#9016) 2020-10-02 16:50:12 -04:00
Javier Heredia 103ac0a37f
Add consul segment fingerprint (#7214) 2020-10-02 15:15:59 -04:00
Tim Gross 56a3e910ea
docs: add Consul policy for Consul Namespace support (#9014)
Co-authored-by: Seth Hoenig <shoenig@hashicorp.com>
2020-10-02 15:13:33 -04:00
Charlie Voiselle c0eca08bcc
Update docker.mdx (#8913) 2020-10-02 14:54:14 -04:00
Fredrik Hoem Grelland a015c52846
configure nomad cluster to use a Consul Namespace [Consul Enterprise] (#8849) 2020-10-02 14:46:36 -04:00
Seth Hoenig fb63d7cad1
Merge pull request #9009 from hashicorp/d-docker-signal-docs
docs: document docker signal fix, add tests
2020-10-02 13:22:15 -05:00
Ryan Oaks 536aab271d Format docs website code and mdx 2020-10-02 13:31:40 -04:00
Kent 'picat' Gruber b03f79700c Fix panic in test due to the agent's logger not being initialized yet
So a null logger is used to avoid the problem.
2020-10-02 11:10:27 -04:00
Seth Hoenig a8869bd304 docs: document docker signal fix, add tests
This PR adds a version specific upgrade note about the docker stop
signal behavior. Also adds test for the signal logic in docker driver.

Closes #8932 which was fixed in #8933
2020-10-02 10:06:43 -05:00
Jean-Baptiste Besselat ffd085b3d9 [doc] fix deprecated port_map example 2020-10-02 14:39:48 +02:00
Tim Gross b6292528fe
e2e: tfvars.dev file must override default tfvars file (#9005)
The `-var-file` flag for loading variables into Terraform overlays the default
variables file if present. This means that variables that are set in the
default variables file will take precedence if the overlay file does not have
them set.

Set `nomad_acls` and `nomad_enteprise` to `false` in the dev cluster.
2020-10-02 08:02:37 -04:00
Fredrik Hoem Grelland 953d4de8dd
update consul-template to v0.25.1 (#8988) 2020-10-01 14:08:49 -04:00
Mahmood Ali 619f5c5274
Merge pull request #9003 from hashicorp/b-tests-vagrant-20200930
Tests: fix some tests for Ubuntu 18.04
2020-10-01 10:45:04 -05:00
Kent 'picat' Gruber 90e85f9add Fix other usages of initKeyring func to use logger as third argument 2020-10-01 11:13:06 -04:00
Kent 'picat' Gruber b98bb99dfe Log AES-128 and AES-192 key sizes during keyring initialization 2020-10-01 11:12:14 -04:00
José Maia 9e14dc90f9
Fix docs on API for volume detaching (#9002)
`nomad volume detach volume-id 00000000-0000-0000-0000-000000000000` produces an API call containing the UUID as part of the query string. This is the only way the API accepts the request correctly - if you pass it in the payload you get `detach requires node ID`
2020-10-01 11:01:13 -04:00
Mahmood Ali f4450db775 tests: use system path
On host with systemd-resolved, we copy /run/systemd/resolve/resolv.conf
actually.
2020-10-01 10:23:19 -04:00
Mahmood Ali 567597e108 Compare to the correct host network setting
In systemd-resolved hosts with no DNS customizations, the docker driver
DNS setting should be compared to /run/systemd/resolve/resolv.conf while
exec/java drivers should be compared to /etc/resolv.conf.

When system-resolved is enabled, /etc/resolv.conf is a stub that points
to 127.0.0.53. Docker avoids this stub because this address isn't
accessible from the container.  The exec/java drivers that don't create
network isolations use the stub though in the default configuration.
2020-10-01 10:23:14 -04:00
Mahmood Ali f4b0aa0c1c tests: copy permissions when copying files
On the failover path, copy the permission bits (a.k.a. file mode),
specially the execution bit.
2020-10-01 10:23:14 -04:00
Mahmood Ali cd060db42a tests: ignore empty cgroup
My latest Vagrant box contains an empty cgroup name that isn't used for
isolation:

```
$ cat /proc/self/cgroup  | grep ::
0::/user.slice/user-1000.slice/session-17.scope
```
2020-10-01 10:23:13 -04:00
Tim Gross 7cf000f64e
changelog: azure fingerprint (#9000) 2020-10-01 09:23:52 -04:00
Landan Cheruka 3df1802119
client: added azure fingerprinting support (#8979) 2020-10-01 09:10:27 -04:00
Kent 'picat' Gruber ef36bcfc43 Update nomad operator keygen example command in docs 2020-09-30 17:07:31 -04:00
Kent 'picat' Gruber 42bdb03f43 Fix operator keygen test to check for 32 bytes 2020-09-30 17:04:33 -04:00
Kent 'picat' Gruber 4570952fe4 Update server configuration docs to use 32 bytes 2020-09-30 17:03:12 -04:00
Kent 'picat' Gruber 6cefe03359 Generate 32-byte gossip key for nomad operator keygen command
The key generated from this command is used for gossip encrpytion, which utilizes AES GCM encryption. Using a key size of 16-bytes enables AES-128 while a key size of 32 bytes enables AES-256.

The underlying memberlist library supports the larger key size, and is ultimatley preferable from a security standpoint. Consul also uses 32 bytes by default: 1a14b94441
2020-09-30 17:02:37 -04:00
Michael Schurter fbcfccfc9e
Merge pull request #8996 from hashicorp/f-lower-mem-min
jobspec: lower min cpu resources from 10->1
2020-09-30 12:35:54 -07:00
Michael Schurter 765473e8b0 jobspec: lower min cpu resources from 10->1
Since CPU resources are usually a soft limit it is desirable to allow
setting it as low as possible to allow tasks to run only in "idle" time.

Setting it to 0 is still not allowed to avoid potential unintentional
side effects with allowing a zero value. While there may not be any side
effects this commit attempts to minimize risk by avoiding the issue.

This does *not* change the defaults.
2020-09-30 12:15:13 -07:00
Mahmood Ali eb5ff115a5
Merge pull request #8992 from hashicorp/b-tests-failover-copy
tests: failover to copying when symlinking fails
2020-09-30 09:22:24 -05:00
Mahmood Ali 91376cccf2 tests: failover to copying when symlinking fails
Symlinking busybox may fail when the test code and the test temporary
directory live on different volumes/partitions; so we should copy
instead.  This situation arises in the Vagrant setup, where the code
repository live on special file sharing volume.

Somewhat unrelated, remove `f.Sync()` invocation from a test copyFile
helper function.  Sync is useful only for crash recovery, and isn't
necessary in our test setup.  The sync invocation is a significant
overhead as it requires the OS to flush any cached writes to disk.
2020-09-30 09:58:22 -04:00
Tim Gross 4bab91b81b
e2e: ensure tests are constrained to Linux (#8990)
Until we have LCOW support in the E2E environment (which requires a Windows
2019 test target), we need to constrain E2E tests to the appropriate kernel
2020-09-30 09:43:30 -04:00
Tim Gross e49410e97b
e2e: cleanup errors should use assert, not require (#8989)
The E2E framework wraps testify's `require` so that by default we can stop
tests on errors, but the cleanup functions should use `assert` so that we
continue to try to cleanup the test environment even if there's a failure.
2020-09-30 09:00:37 -04:00
Lars Lehtonen 03abe3c890
client: fix test umask (#8987) 2020-09-30 08:09:41 -04:00
Luiz Aoqui 32c6a99ff3
Merge pull request #8985 from hashicorp/f-scaling-policy-type
Add scaling policy type
2020-09-29 18:18:24 -04:00