Commit graph

2830 commits

Author SHA1 Message Date
Tim Gross de6b023af2 command: remove -namespace from help options when not applicable 2020-11-19 16:28:39 -05:00
Tim Gross d67afa2e21 docs/help: -no-color does not apply to alloc logs content
The `nomad alloc logs` command does not remove terminal escape sequences for
color from the log outputs of a task. Clarify that the standard `-no-color`
flag, which does apply to Nomad's error responses from `nomad alloc logs`,
does not apply to the log output.
2020-11-19 15:29:12 -05:00
Chris Baker afe6fa82c2
Merge pull request #9368 from hashicorp/b-9367-duped-ports
fix duplication of ports in `AllocatedResources`
2020-11-16 11:41:34 -06:00
Tim Gross a788557579
csi: alloc status -verbose should query volume request 'source' (#9354)
The `nomad alloc status -verbose` command returns a 404 from CSI volumes
because the volume mount block in the task points back to the
`job.group.volume` block. So using the `Name` field to query is the "name" as
seen in the jobspec, and not the name of the volume that we need for querying.

Show both the job-specific name and the volume ID in the resulting output,
which clarifies the difference between the two fields and is more consistent
with the web UI.
2020-11-16 08:28:52 -05:00
Chris Baker 7c6071e6c4 updated alloc_endpoint to mutate a copy of the returned allocation, instead of the instance in the state store 2020-11-15 17:52:50 +00:00
Chris Baker b244d5e949 documenting test for #9367 2020-11-15 17:47:50 +00:00
Seth Hoenig 4cc3c01d5b
Merge pull request #9352 from hashicorp/f-artifact-headers
jobspec: add support for headers in artifact stanza
2020-11-13 14:04:27 -06:00
Seth Hoenig bb8a5816a0 jobspec: add support for headers in artifact stanza
This PR adds the ability to set HTTP headers when downloading
an artifact from an `http` or `https` resource.

The implementation in `go-getter` is such that a new `HTTPGetter`
must be created for each artifact that sets headers (as opposed
to conveniently setting headers per-request). This PR maintains
the memoization of the default Getter objects, creating new ones
only for artifacts where headers are set.

Closes #9306
2020-11-13 12:03:54 -06:00
Dave May e89302aa4b
nomad operator debug - add client node filtering arguments (#9331)
* operator debug - add client node filtering arguments

* add WaitForClient helper function

* use RPC in WaitForClient to avoid unnecessary imports

* guard against nil values

* move initialization up and shorten test duration

* cleanup nodeLookupFailCount logic

* only display max node notice if we actually tried to capture nodes
2020-11-12 11:25:28 -05:00
Seth Hoenig b19bc6be2b consul: prevent re-registration churn by correctly comparing sidecar tags
Previously, connect sidecars would be re-registered with consul every cycle
of Nomad's reconciliation loop around Consul service registrations. This is
because part of the comparison used `reflect.DeepEqual` on []string objects,
which returns false when one object is `[]string{}` and the other is `[]string{}(nil)`.

Unforunately, this was always the case, and every Connect sidecar service
would be re-registered on every iteration, which happens every 30 seconds.
2020-11-11 18:01:17 -06:00
Chris Baker 611502dd5f
Merge pull request #9327 from hashicorp/b-fix-rec-cli
recs cli: fixed bad composition
2020-11-11 15:54:14 -06:00
Nick Ethier 5e1634eda1
structs: canonicalize allocatedtaskresources to populate shared ports (#9309) 2020-11-11 16:21:47 -05:00
Chris Baker bc82312eb4 recs cli: fixed bad composition 2020-11-11 20:52:57 +00:00
Tim Gross 60874ebe25
csi: Postrun hook should not change mode (#9323)
The unpublish workflow requires that we know the mode (RW vs RO) if we want to
unpublish the node. Update the hook and the Unpublish RPC so that we mark the
claim for release in a new state but leave the mode alone. This fixes a bug
where RO claims were failing node unpublish.

The core job GC doesn't know the mode, but we don't need it for that workflow,
so add a mode specifically for GC; the volumewatcher uses this as a sentinel
to check whether claims (with their specific RW vs RO modes) need to be claimed.
2020-11-11 13:06:30 -05:00
Chris Baker 2deb77dcf5 cli: updated recommendation commands and test to remove duplication of autocompletion code 2020-11-11 11:39:26 +00:00
Chris Baker e3c0ea654d auto-complete for recommendations CLI, plus OSS components of recommendations prefix search 2020-11-11 11:13:43 +00:00
Mahmood Ali f61d284fe6 Custom message when job file is HCL2 incompatible
Use a custom message when the job file is a valid HCL1 but no longer
valid under HCL 2 syntax.
2020-11-10 14:56:20 -05:00
Mahmood Ali 69849a42a5
Merge pull request #9298 from hashicorp/f-hcl2-localsvars
HCL2: Variables and Locals
2020-11-09 16:44:37 -05:00
Luiz Aoqui ea81ac5d3d
Merge pull request #9296 from hashicorp/b-remove-namespace-from-scale-request
Remove Namespace field from JobScaleRequest
2020-11-09 15:13:33 -05:00
Mahmood Ali 7fcda0f01f Start accepting input variable files as CLI arguments 2020-11-09 15:02:58 -05:00
Mahmood Ali 1ae3e8a8eb Start using the new jobspec2 API 2020-11-09 15:01:31 -05:00
Luiz Aoqui c536286c7a
remove Namespace field from JobScaleRequest 2020-11-09 13:02:05 -05:00
James Rasell 473fb4bb17
Merge pull request #9272 from hashicorp/f-recommendation-cli
cli: add recommendation commands.
2020-11-06 16:29:57 +01:00
James Rasell fe92d8b3cb
cli: add recommendation commands.
Adds new CLI commands for applying, dismissing and detailing Nomad
recommendations under a new top level recommendation command.
2020-11-06 11:16:24 +01:00
Nick Ethier 04f5c4ee5f
ar/groupservice: remove drivernetwork (#9233)
* ar/groupservice: remove drivernetwork

* consul: allow host address_mode to accept raw port numbers

* consul: fix logic for blank address
2020-11-05 15:00:22 -05:00
Charlie Voiselle 0e373f70c1
Merge pull request #9255 from hashicorp/d-missing-example-comma
Add missing comma in help-text example.
2020-11-04 09:51:09 -05:00
James Rasell b147ec1e57
cli: update scaling policy list help to include job flag. 2020-11-04 13:35:29 +01:00
Charlie Voiselle 443a93be11
Add missing comma in help-text example.
@krishicks spotted this while playing with command.
2020-11-02 18:00:53 -05:00
Kris Hicks 48a260fc33
Update monitor func not to take a prefix (#9251)
The only user of monitor(evalID, true) was command/eval_status, and
eval_status had a duplicate of the prefix-handling code inside it, so in
all cases the complete evalID was being passed to monitor.

Given that, we can remove the prefix code from command/monitor, and
remove the boolean arg.
2020-11-02 10:24:49 -08:00
Kris Hicks 1da9e7fc67
Add event sink API and CLI commands (#9226)
Co-authored-by: Drew Bailey <2614075+drewbailey@users.noreply.github.com>
2020-11-02 09:57:35 -08:00
Chris Baker 719077a26d added new policy capabilities for recommendations API
state store: call-out to generic update of job recommendations from job update method
recommendations API work, and http endpoint errors for OSS
support for scaling polices in task block of job spec
add query filters for ScalingPolicy list endpoint
command: nomad scaling policy list: added -job and -type
2020-10-28 14:32:16 +00:00
Drew Bailey 86080e25a9
Send events to EventSinks (#9171)
* Process to send events to configured sinks

This PR adds a SinkManager to a server which is responsible for managing
managed sinks. Managed sinks subscribe to the event broker and send
events to a sink writer (webhook). When changes to the eventstore are
made the sinkmanager and managed sink are responsible for reloading or
starting a new managed sink.

* periodically check in sink progress to raft

Save progress on the last successfully sent index to raft. This allows a
managed sink to resume close to where it left off in the event of a lost
server or leadership change

dereference eventsink so we can accurately use the watchch

When using a pointer to eventsink struct it was updated immediately and our reload logic would not trigger
2020-10-26 17:27:54 -04:00
Drew Bailey 1ae39a9ed9
event sink crud operation api (#9155)
* network sink rpc/api plumbing

state store methods and restore

upsert sink test

get sink

delete sink

event sink list and tests

go generate new msg types

validate sink on upsert

* go generate
2020-10-23 14:23:00 -04:00
Michael Schurter c2dd9bc996 core: open source namespaces 2020-10-22 15:26:32 -07:00
Mahmood Ali 059e87c862
Merge pull request #9142 from hashicorp/f-hclv2-2.3
Support HCLv2 for Nomad jobs
2020-10-22 12:26:28 -05:00
Drew Bailey f3dcefe5a9
remove event durability (#9147)
* remove event durability

temporarily removing go-memdb event durability until a new strategy is developed on how to best handled increased durability needs

* drop events table schema and state store methods

* fix neweventbuffer invocations
2020-10-22 12:21:03 -04:00
Mahmood Ali d3a17b5c82 address review feedback 2020-10-22 11:49:37 -04:00
Mahmood Ali f52bda4c30 api: update /render api to parse hclv2 2020-10-21 15:46:57 -04:00
Mahmood Ali 84ec0b38e8 cli: use HCLv2 parser
Also, fallback to using HCLv1.
2020-10-21 15:46:57 -04:00
Mahmood Ali 1ae924973e hclv1: tweak HCLv1 tests
This ensures that gatway ReadOnly key is tested.  Also, update the hclv1
test-fixtures to be hclv1 compliant.
2020-10-21 14:05:46 -04:00
Drew Bailey 6c788fdccd
Events/msgtype cleanup (#9117)
* use msgtype in upsert node

adds message type to signature for upsert node, update tests, remove placeholder method

* UpsertAllocs msg type test setup

* use upsertallocs with msg type in signature

update test usage of delete node

delete placeholder msgtype method

* add msgtype to upsert evals signature, update test call sites with test setup msg type

handle snapshot upsert eval outside of FSM and ignore eval event

remove placeholder upsertevalsmsgtype

handle job plan rpc and prevent event creation for plan

msgtype cleanup upsertnodeevents

updatenodedrain msgtype

msg type 0 is a node registration event, so set the default  to the ignore type

* fix named import

* fix signature ordering on upsertnode to match
2020-10-19 09:30:15 -04:00
Drew Bailey fba0d6dc6a
event buffer size and durable count must be non negative 2020-10-15 16:34:33 -04:00
Nick Ethier 4903e5b114
Consul with CNI and host_network addresses (#9095)
* consul: advertise cni and multi host interface addresses

* structs: add service/check address_mode validation

* ar/groupservices: fetch networkstatus at hook runtime

* ar/groupservice: nil check network status getter before calling

* consul: comment network status can be nil
2020-10-15 15:32:21 -04:00
Michael Schurter ea55c497b7
Merge pull request #9094 from hashicorp/f-1.0
s/0.13/1.0/g
2020-10-15 08:53:33 -07:00
James Rasell 42a6e7140f
Merge pull request #9083 from hashicorp/b-fix-enterprise-config-merge
agent: fix enterprise config overlay merging.
2020-10-15 08:40:49 +02:00
Michael Schurter 9c3972937b s/0.13/1.0/g
1.0 here we come!
2020-10-14 15:17:47 -07:00
Michael Schurter dd09fa1a4a
Merge pull request #9055 from hashicorp/f-9017-resources
api: add field filters to /v1/{allocations,nodes}
2020-10-14 14:49:39 -07:00
Michael Schurter 6890cffd7a unify boolean parameter parsing 2020-10-14 12:23:25 -07:00
Dave May f37e90be18
Metrics gotemplate support, debug bundle features (#9067)
* add goroutine text profiles to nomad operator debug

* add server-id=all to nomad operator debug

* fix bug from changing metrics from string to []byte

* Add function to return MetricsSummary struct, metrics gotemplate support

* fix bug resolving 'server-id=all' when no servers are available

* add url to operator_debug tests

* removed test section which is used for future operator_debug.go changes

* separate metrics from operator, use only structs from go-metrics

* ensure parent directories are created as needed

* add suggested comments for text debug pprof

* move check down to where it is used

* add WaitForFiles helper function to wait for multiple files to exist

* compact metrics check

Co-authored-by: Drew Bailey <2614075+drewbailey@users.noreply.github.com>

* fix github's silly apply suggestion

Co-authored-by: Drew Bailey <2614075+drewbailey@users.noreply.github.com>
2020-10-14 15:16:10 -04:00
Drew Bailey c463479848
filter on additional filter keys, remove switch statement duplication
properly wire up durable event count

move newline responsibility

moves newline creation from NDJson to the http handler, json stream only encodes and sends now

ignore snapshot restore if broker is disabled

enable dev mode to access event steam without acl

use mapping instead of switch

use pointers for config sizes, remove unused ttl, simplify closed conn logic
2020-10-14 14:14:33 -04:00