Commit Graph

21309 Commits

Author SHA1 Message Date
Seth Hoenig 6e1c71446d client: always set script checks hook
Similar to a bugfix made for the services hook, we need to always
set the script checks hook, in case a task is initially launched
without script checks, but then updated to include script checks.

The scipt checks hook is the thing that handles that new registration.
2021-04-19 15:37:42 -06:00
Seth Hoenig 509490e5d2 e2e: consul namespace tests from nomad ent
(cherry-picked from ent without _ent things)

This is part 2/4 of e2e tests for Consul Namespaces. Took a
first pass at what the parameterized tests can look like, but
only on the ENT side for this PR. Will continue to refactor
in the next PRs.

Also fixes 2 bugs:
 - Config Entries registered by Nomad Server on job registration
   were not getting Namespace set
 - Group level script checks were not getting Namespace set

Those changes will need to be copied back to Nomad OSS.

Nomad OSS + no ACLs (previously, needs refactor)
Nomad ENT + no ACLs (this)
Nomad OSS + ACLs (todo)
Nomad ENT + ALCs (todo)
2021-04-19 15:35:31 -06:00
Seth Hoenig 9c4e2e93d0 deps: update consul api for transparent proxy 2021-04-19 14:56:01 -06:00
Nick Ethier 8140b0160c
Merge pull request #10369 from hashicorp/f-cpu-cores-4
Reserved Cores [4/4]: Implement driver cpuset cgroup path consumption
2021-04-19 14:53:29 -04:00
Mahmood Ali 7e99c49e37
Merge pull request #10397 from hashicorp/f-cli-file-arg
cli: filename arg for `volume init` and `quote init`
2021-04-19 11:02:49 -04:00
Mahmood Ali d880ba9c62 cli: filename arg for `volume init` and `quote init` 2021-04-18 14:14:05 -04:00
Seth Hoenig c34ef9eb78 api: fuzzy search results include job name with id in scope 2021-04-16 17:03:36 -06:00
Seth Hoenig 0b2114a7a5 api: make fuzzy searching case-agnostic 2021-04-16 16:56:10 -06:00
Seth Hoenig 1ee8d5ffc5 api: implement fuzzy search API
This PR introduces the /v1/search/fuzzy API endpoint, used for fuzzy
searching objects in Nomad. The fuzzy search endpoint routes requests
to the Nomad Server leader, which implements the Search.FuzzySearch RPC
method.

Requests to the fuzzy search API are based on the api.FuzzySearchRequest
object, e.g.

{
  "Text": "ed",
  "Context": "all"
}

Responses from the fuzzy search API are based on the api.FuzzySearchResponse
object, e.g.

{
  "Index": 27,
  "KnownLeader": true,
  "LastContact": 0,
  "Matches": {
    "tasks": [
      {
        "ID": "redis",
        "Scope": [
          "default",
          "example",
          "cache"
        ]
      }
    ],
    "evals": [],
    "deployment": [],
    "volumes": [],
    "scaling_policy": [],
    "images": [
      {
        "ID": "redis:3.2",
        "Scope": [
          "default",
          "example",
          "cache",
          "redis"
        ]
      }
    ]
  },
  "Truncations": {
    "volumes": false,
    "scaling_policy": false,
    "evals": false,
    "deployment": false
  }
}

The API is tunable using the new server.search stanza, e.g.

server {
  search {
    fuzzy_enabled   = true
    limit_query     = 200
    limit_results   = 1000
    min_term_length = 5
  }
}

These values can be increased or decreased, so as to provide more
search results or to reduce load on the Nomad Server. The fuzzy search
API can be disabled entirely by setting `fuzzy_enabled` to `false`.
2021-04-16 16:36:07 -06:00
Seth Hoenig 650b94f89f
Merge pull request #10387 from hashicorp/e2e-consul-namespaces-oss
e2e: add tests for consul namespaces from nomad oss
2021-04-16 16:34:09 -06:00
Seth Hoenig 25810b4cd6 e2e: set PORT on counter-api with host networking 2021-04-16 16:28:39 -06:00
Seth Hoenig 2d693127bb e2e: minor tweaks from CR 2021-04-16 15:32:37 -06:00
Seth Hoenig 7f1191111d e2e: add tests for consul namespaces from nomad oss
This PR adds a set of tests to the Consul test suite for testing
Nomad OSS's behavior around setting Consul Namespace on groups,
which is to ignore the setting (as Consul Namespaces are currently
an Enterprise feature).

Tests are generally a reduced facsimile of existing tests, modified
to check behavior of when group.consul.namespace is set and not set.
Verification is oriented around what happens in Consul; the in-depth
functional correctness of these features is left to the original tests.

Nomad ENT will get its own version of these tests in `namespaces_ent.go`.
2021-04-16 15:32:37 -06:00
Seth Hoenig b6aa6d11b5
Merge pull request #10393 from hashicorp/deps-revert-consul-api-bump
Revert "deps: bump consul/api version for transparent proxy"
2021-04-16 15:32:07 -06:00
Nick Ethier b34db8b3b6 nit: code cleanup/organization 2021-04-16 15:14:29 -04:00
Seth Hoenig f7251368fa Revert "deps: bump consul/api version for transparent proxy"
This reverts commit 12bb509242109770c8318ec8ca61e54f4fa2bd42.

This change is causing compatibility issues with Consul 1.9.X;
I suspect the change is only compatible with Consul 1.10 or higher
which is not released yet, and not something Nomad can require
for quite some time.

E.g. when registering an ingress gateway,

```
consul.sync: failed to update services in Consul: error="Unexpected response code: 400 (Request decode failed: json: unknown field "TransparentProxy")"
```
2021-04-16 13:13:56 -06:00
Nick Ethier cde820f14b update go.mod 2021-04-16 14:56:07 -04:00
Nick Ethier 110f982eb3 plugins/drivers: fix deprecated fields 2021-04-16 14:13:29 -04:00
Nick Ethier f6d7285157
Merge pull request #10328 from hashicorp/f-cpu-cores-3
Reserved Cores [3/4]: Client cpuset cgroup managment
2021-04-16 14:11:45 -04:00
Nick Ethier 86fb1f156a
Merge pull request #10228 from hashicorp/f-cpu-cores-2
Reserved Cores [2/4]: Client fingerprinting implementation
2021-04-16 14:10:25 -04:00
Shishir Mahajan 86e2a2be4f Update containerd task driver options.
- image_pull_timeout
- pids_limit
- sysctl
2021-04-16 13:18:33 -04:00
Nick Ethier 07dca26f0d qemu: set the number of cores equal to the number of reserved cores if set 2021-04-15 13:32:33 -04:00
Nick Ethier 1e09ca5cd7 tr: set cpuset cpus if reserved 2021-04-15 13:31:51 -04:00
Nick Ethier db1f697fc0 plugins/driver: add cpuset_cpus back and mark cpuset_mems as reserved 2021-04-15 13:31:18 -04:00
Seth Hoenig 4d9766a758
Merge pull request #10377 from hashicorp/d-bump-consulapi
deps: bump consul/api version for transparent proxy
2021-04-15 09:13:36 -06:00
Tim Gross dcc5268862 E2E/CSI: ensure jobs are stopped before checking claims are released
During refactoring of the CSI jobs, the EBS test dropped stopping the jobs
before checking that the claims were released.
2021-04-15 11:06:11 -04:00
Seth Hoenig c85dbc014f
Merge pull request #10382 from hashicorp/e2e-use-consul-ent
e2e: get consul ent in e2e packer builds
2021-04-15 08:56:51 -06:00
Nick Ethier c9216ba7d9 drivers/docker: move cgroups logic to linux build file 2021-04-15 10:39:11 -04:00
Nick Ethier 390c4c5119 docker: add support for cpuset cgroup management 2021-04-15 10:24:31 -04:00
Nick Ethier fe283c5a8f executor: add support for cpuset cgroup 2021-04-15 10:24:31 -04:00
Nick Ethier 45aee28c03 cgutil: set reserved mems on init even if already exist 2021-04-15 10:24:31 -04:00
Tim Gross 0518552e4d docs: changelog and upgrade note for iptables improvement 2021-04-15 10:19:37 -04:00
Adam Duncan 7588cf0ec3 networking: Ensure CNI iptables rules are appended to chain and not forced to be first 2021-04-15 10:11:15 -04:00
changli0617 5723e194c1
Update partnerships.mdx
Need help to resize the ecosystem diagram.
2021-04-14 22:22:35 -07:00
Mike Wickett e1c511645f website: stub in partnerships page 2021-04-14 19:55:45 -04:00
Nick Ethier b235091a51 client: disable cpuset cgroup managment if init fails 2021-04-14 14:44:08 -04:00
Seth Hoenig 198e0d9f24 e2e: get consul ent in e2e packer builds
Using Consul Enterprise is going to be necessary for testing Nomad's
Consul Namespace integration in Nomad v1.1 in e2e.
2021-04-14 12:05:55 -06:00
Mahmood Ali a239224555
Merge pull request #10381 from hashicorp/f-cv-fix-missing-var-msg
Fix link in error for missing variable
2021-04-14 13:02:22 -04:00
Charlie Voiselle 2386c1348e
Fix link in error for missing variable 2021-04-14 12:55:00 -04:00
Nick Ethier d7ab0b8a86 another testing fix 2021-04-14 10:37:03 -04:00
Nick Ethier 0a4e298221 testing fixes 2021-04-14 10:17:28 -04:00
Michael Lange 16de29da8f
Merge pull request #10288 from hashicorp/b-ui/no-downloads-in-test
UI: No downloads in test
2021-04-13 16:51:16 -07:00
Michael Lange 05f4c5d0bd Stop downloading files in tests 2021-04-13 16:40:17 -07:00
Michael Lange 75ca53c7ea Cleanup: Use this.attr in test templates 2021-04-13 16:40:17 -07:00
Nick Ethier 6f3fe8a11e cgutil: add nil check on AddAlloc 2021-04-13 13:28:36 -04:00
Nick Ethier 155a2ca5fb client/ar: thread through cpuset manager 2021-04-13 13:28:36 -04:00
Nick Ethier b6b74a98a9 client/fingerprint: move existing cgroup concerns to cgutil 2021-04-13 13:28:36 -04:00
Nick Ethier bc7df41914 lib/cpuset: add String and ContainsAny helpers 2021-04-13 13:28:36 -04:00
Nick Ethier 411d992788 cgutil: implement cpuset management as seperate package 2021-04-13 13:28:36 -04:00
Nick Ethier 0a21de91dd Apply suggestions from code review
Co-authored-by: Drew Bailey <drewbailey5@gmail.com>
2021-04-13 13:28:15 -04:00