Commit graph

21291 commits

Author SHA1 Message Date
James Rasell 2dfdf99f91
Merge pull request #10448 from hashicorp/f-docs-add-autoscaler-plugin-launch-info
docs: add detail on launching autoscaler external plugins.
2021-04-27 08:29:31 +02:00
James Rasell 4e18e9ea8b
docs: add detail on launching autoscaler external plugins. 2021-04-26 11:19:15 +02:00
Mahmood Ali 2a66922daf
Merge pull request #10442 from kphunter/patch-1
Fix path tip
2021-04-24 08:34:59 -04:00
kphunter 59209f4165
Fix path tip 2021-04-23 22:03:53 -07:00
Mahmood Ali cf24a9eaaf
api: /v1/jobs always include namespaces (#10434)
Add Namespace as a top-level field in `/v1/jobs` stub.

The `/v1/jobs` endpoint already includes the namespace under `JobSummary`, though the API is odd, as typically the job ID and Namespace are in the same level, and the oddity complicates the UI frontend development.

The downside of adding it is redundant field, that makes the response body a bit bigger, specially for clusters with large jobs. Though, it should compress nicely and I expect the overhead to be small to overall response size. The benefit of a cleaner and more consistent API seem worth it.

Fixes #10431
2021-04-23 16:36:54 -04:00
Mahmood Ali ed4aad458c
api: Ignore User provided ParentID (#10424)
ParentID is an internal field that Nomad sets for dispatched or parameterized jobs. Job submitters should not be able to set it directly, as that messes up children tracking.

Fixes #10422 . It specifically stops the scheduler from honoring the ParentID. The reason failure and why the scheduler didn't schedule that job once it was created is very interesting and requires follow up with a more technical issue.
2021-04-23 16:22:17 -04:00
Luiz Aoqui 2011e15d63
changelog: add entry for #10348 (#10349) 2021-04-23 11:25:31 -04:00
Zachary Shilton b1bd139c3c
website: add check for unlinked content (#10425)
* website: add unlinked content check

* website: remove unused guides-navigation.js
2021-04-23 11:11:09 -04:00
Luiz Aoqui 29171be859
docs: reorganize autoscaling agent config (#10383) 2021-04-23 09:53:58 -04:00
changli0617 5c820bae59
Small typo fixes (#10427)
Co-authored-by: Charlie Voiselle <464492+angrycub@users.noreply.github.com>
2021-04-22 12:16:21 -07:00
changli0617 b74cb407bd
Merge pull request #10384 from hashicorp/mw.partnership-page
website: add partnerships page
2021-04-22 11:38:05 -07:00
Seth Hoenig 25d5cebd15
Merge pull request #10426 from hashicorp/up-aws-instances
aws_env: update ec2 instances
2021-04-22 12:19:27 -06:00
Seth Hoenig f53c30c684 aws_env: update ec2 instances
Generate updated list using tools/ec2info
2021-04-22 11:33:51 -06:00
Zachary Shilton 8dd97f5f76
website: update readme (#10420)
* website: bump to latest nextjs-scripts prerelease

* website: run generate:readme to update readme blocks

* website: revert bump to nextjs-scripts, deferred
2021-04-22 13:08:26 -04:00
changli0617 d80d7229d5
Update partnerships.mdx 2021-04-21 15:20:04 -07:00
Buck Doyle 2a72920af1
docs: Fix minor fuzzy search things (#10423) 2021-04-21 14:55:17 -05:00
Mahmood Ali 2d6c75a17f
hcl2: handle unquoted undefined variables (#10419)
This fixes a regression in #10326, to handle unquoted unknown variables.

The HCL job may contain unquoted undefined variable references without ${...} wrapping, e.g. value = meta.node_class. In 1.0.4, this got parsed as value = "${meta.node_class}".

This code performs a scan to find the relevant ${ and }, and only tries to find the closest ones with whitespace as the only separator.
2021-04-21 13:24:22 -04:00
Mahmood Ali d2fcce21f8
Migrate all allocs when draining a node (#10411)
This fixes a bug affecting drain nodes, where allocs may fail to be
migrated if they belong to different namespaces but share the same job
name.

The reason is that the helper function that creates the migration evals
indexed the allocs by job ID without accounting for the namespaces.
When job ids clash, only an eval is created for one and the rest of the
allocs remain intact.

Fixes #10172
2021-04-21 12:11:14 -04:00
James Rasell 02ee9f605e
Merge pull request #10409 from hashicorp/f-docs-gh-10406
docs: update autoscaler target plugins to include datacenter field.
2021-04-21 10:15:45 +02:00
Charlie Voiselle 5d9e4fcf25
Add PR 10404 to CHANGELOG (#10415) 2021-04-20 14:01:22 -04:00
Charlie Voiselle ef8ca60693
Enable go-sockaddr templating for network-interface (#10404)
Add templating to `network-interface` option.
This PR also adds a fast-fail to in the case where an invalid interface is set or produced by the template

* add tests and check for valid interface
* Add documentation
* Incorporate suggestions from code review

Co-authored-by: Luiz Aoqui <luiz@hashicorp.com>
2021-04-20 13:55:10 -04:00
Seth Hoenig d86bf9b625
Merge pull request #10413 from hashicorp/up-fuzzy-ent-structs
api: include ent fuzzy struct types in oss
2021-04-20 11:41:34 -06:00
Seth Hoenig f71dd3857e api: include ent fuzzy struct types in oss
Small change to pull in ent struct types in a switch
statement used by ent. They are benign in oss, this
is just to make sure OSS->ENT merges don't create a
diff.
2021-04-20 11:19:38 -06:00
Seth Hoenig 4e6dbaaec1
Merge pull request #10184 from hashicorp/f-fuzzy-search
api: implement fuzzy search API
2021-04-20 09:06:40 -06:00
Seth Hoenig f258fc8270
Merge pull request #10401 from hashicorp/cp-cns-ent-test-fixes
cherry-pick fixes from cns ent tests
2021-04-20 08:45:15 -06:00
Seth Hoenig 5448fe2029
Merge pull request #10402 from hashicorp/deps-update-consul-api
deps: update consul api for transparent proxy
2021-04-20 08:43:24 -06:00
Buck Doyle 90ecbdf522
Add job version revert buttons (#10336)
This adds a Revert two-step button to the JobVersions component for
not-current versions, which redirects to the overview on success. It
checks the job version before and after reversion to mitigate the edge
case where reverting to an otherwise-identical version has no effect, as
discussed in #10337.

It uses existing facilities for handling other errors and disabling the
button when permissions are lacking.
2021-04-20 08:33:16 -05:00
Drew Bailey d42f204a89
remove second deploy that did not have anything to do with the test (#10400) 2021-04-20 08:44:44 -04:00
James Rasell d55ae5d194
docs: update autoscaler target plugins to include datacenter field. 2021-04-20 14:01:19 +02:00
James Rasell e8f271ae09
Merge pull request #10408 from jsiebens/patch-1
website: added digitalocean droplets plugin to community plugins index
2021-04-20 10:58:18 +02:00
Johan Siebens 8f358838e2
website: added digitalocean droplets plugin to community plugins index 2021-04-20 10:25:21 +02:00
Seth Hoenig 49befcb8e0
Merge pull request #10403 from hashicorp/b-always-set-scriptchecks-hook
client: always set script checks hook
2021-04-19 16:13:45 -06:00
Seth Hoenig 6e1c71446d client: always set script checks hook
Similar to a bugfix made for the services hook, we need to always
set the script checks hook, in case a task is initially launched
without script checks, but then updated to include script checks.

The scipt checks hook is the thing that handles that new registration.
2021-04-19 15:37:42 -06:00
Seth Hoenig 509490e5d2 e2e: consul namespace tests from nomad ent
(cherry-picked from ent without _ent things)

This is part 2/4 of e2e tests for Consul Namespaces. Took a
first pass at what the parameterized tests can look like, but
only on the ENT side for this PR. Will continue to refactor
in the next PRs.

Also fixes 2 bugs:
 - Config Entries registered by Nomad Server on job registration
   were not getting Namespace set
 - Group level script checks were not getting Namespace set

Those changes will need to be copied back to Nomad OSS.

Nomad OSS + no ACLs (previously, needs refactor)
Nomad ENT + no ACLs (this)
Nomad OSS + ACLs (todo)
Nomad ENT + ALCs (todo)
2021-04-19 15:35:31 -06:00
Seth Hoenig 9c4e2e93d0 deps: update consul api for transparent proxy 2021-04-19 14:56:01 -06:00
Nick Ethier 8140b0160c
Merge pull request #10369 from hashicorp/f-cpu-cores-4
Reserved Cores [4/4]: Implement driver cpuset cgroup path consumption
2021-04-19 14:53:29 -04:00
Mahmood Ali 7e99c49e37
Merge pull request #10397 from hashicorp/f-cli-file-arg
cli: filename arg for `volume init` and `quote init`
2021-04-19 11:02:49 -04:00
Mahmood Ali d880ba9c62 cli: filename arg for volume init and quote init 2021-04-18 14:14:05 -04:00
Seth Hoenig c34ef9eb78 api: fuzzy search results include job name with id in scope 2021-04-16 17:03:36 -06:00
Seth Hoenig 0b2114a7a5 api: make fuzzy searching case-agnostic 2021-04-16 16:56:10 -06:00
Seth Hoenig 1ee8d5ffc5 api: implement fuzzy search API
This PR introduces the /v1/search/fuzzy API endpoint, used for fuzzy
searching objects in Nomad. The fuzzy search endpoint routes requests
to the Nomad Server leader, which implements the Search.FuzzySearch RPC
method.

Requests to the fuzzy search API are based on the api.FuzzySearchRequest
object, e.g.

{
  "Text": "ed",
  "Context": "all"
}

Responses from the fuzzy search API are based on the api.FuzzySearchResponse
object, e.g.

{
  "Index": 27,
  "KnownLeader": true,
  "LastContact": 0,
  "Matches": {
    "tasks": [
      {
        "ID": "redis",
        "Scope": [
          "default",
          "example",
          "cache"
        ]
      }
    ],
    "evals": [],
    "deployment": [],
    "volumes": [],
    "scaling_policy": [],
    "images": [
      {
        "ID": "redis:3.2",
        "Scope": [
          "default",
          "example",
          "cache",
          "redis"
        ]
      }
    ]
  },
  "Truncations": {
    "volumes": false,
    "scaling_policy": false,
    "evals": false,
    "deployment": false
  }
}

The API is tunable using the new server.search stanza, e.g.

server {
  search {
    fuzzy_enabled   = true
    limit_query     = 200
    limit_results   = 1000
    min_term_length = 5
  }
}

These values can be increased or decreased, so as to provide more
search results or to reduce load on the Nomad Server. The fuzzy search
API can be disabled entirely by setting `fuzzy_enabled` to `false`.
2021-04-16 16:36:07 -06:00
Seth Hoenig 650b94f89f
Merge pull request #10387 from hashicorp/e2e-consul-namespaces-oss
e2e: add tests for consul namespaces from nomad oss
2021-04-16 16:34:09 -06:00
Seth Hoenig 25810b4cd6 e2e: set PORT on counter-api with host networking 2021-04-16 16:28:39 -06:00
Seth Hoenig 2d693127bb e2e: minor tweaks from CR 2021-04-16 15:32:37 -06:00
Seth Hoenig 7f1191111d e2e: add tests for consul namespaces from nomad oss
This PR adds a set of tests to the Consul test suite for testing
Nomad OSS's behavior around setting Consul Namespace on groups,
which is to ignore the setting (as Consul Namespaces are currently
an Enterprise feature).

Tests are generally a reduced facsimile of existing tests, modified
to check behavior of when group.consul.namespace is set and not set.
Verification is oriented around what happens in Consul; the in-depth
functional correctness of these features is left to the original tests.

Nomad ENT will get its own version of these tests in `namespaces_ent.go`.
2021-04-16 15:32:37 -06:00
Seth Hoenig b6aa6d11b5
Merge pull request #10393 from hashicorp/deps-revert-consul-api-bump
Revert "deps: bump consul/api version for transparent proxy"
2021-04-16 15:32:07 -06:00
Nick Ethier b34db8b3b6 nit: code cleanup/organization 2021-04-16 15:14:29 -04:00
Seth Hoenig f7251368fa Revert "deps: bump consul/api version for transparent proxy"
This reverts commit 12bb509242109770c8318ec8ca61e54f4fa2bd42.

This change is causing compatibility issues with Consul 1.9.X;
I suspect the change is only compatible with Consul 1.10 or higher
which is not released yet, and not something Nomad can require
for quite some time.

E.g. when registering an ingress gateway,

```
consul.sync: failed to update services in Consul: error="Unexpected response code: 400 (Request decode failed: json: unknown field "TransparentProxy")"
```
2021-04-16 13:13:56 -06:00
Nick Ethier cde820f14b update go.mod 2021-04-16 14:56:07 -04:00
Nick Ethier 110f982eb3 plugins/drivers: fix deprecated fields 2021-04-16 14:13:29 -04:00