Commit Graph

2778 Commits

Author SHA1 Message Date
Drew Bailey 684807bddb
namespace filtering 2020-10-14 12:44:43 -04:00
Drew Bailey df96b89958
Add EvictCallbackFn to handle removing entries from go-memdb when they
are removed from the event buffer.

Wire up event buffer size config, use pointers for structs.Events
instead of copying.
2020-10-14 12:44:42 -04:00
Drew Bailey 315f77a301
rehydrate event publisher on snapshot restore
address pr feedback
2020-10-14 12:44:41 -04:00
Drew Bailey d793529d61
event durability count and cfg 2020-10-14 12:44:40 -04:00
Drew Bailey b4c135358d
use Events to wrap index and events, store in events table 2020-10-14 12:44:39 -04:00
Drew Bailey 559517455a
wire up enable_event_publisher 2020-10-14 12:44:38 -04:00
Drew Bailey 4793bb4e01
Events/deployment events (#9004)
* Node Drain events and Node Events (#8980)

Deployment status updates

handle deployment status updates (paused, failed, resume)

deployment alloc health

generate events from apply plan result

txn err check, slim down deployment event

one ndjson line per index

* consolidate down to node event + type

* fix UpdateDeploymentAllocHealth test invocations

* fix test
2020-10-14 12:44:37 -04:00
Drew Bailey a4a2975edf
Event Stream API/RPC (#8947)
This Commit adds an /v1/events/stream endpoint to stream events from.

The stream framer has been updated to include a SendFull method which
does not fragment the data between multiple frames. This essentially
treats the stream framer as a envelope to adhere to the stream framer
interface in the UI.

If the `encode` query parameter is omitted events will be streamed as
newline delimted JSON.
2020-10-14 12:44:36 -04:00
Chris Baker 1d35578bed removed backwards-compatible/untagged metrics deprecated in 0.7 2020-10-13 20:18:39 +00:00
Seth Hoenig ed13e5723f consul/connect: dynamically select envoy sidecar at runtime
As newer versions of Consul are released, the minimum version of Envoy
it supports as a sidecar proxy also gets bumped. Starting with the upcoming
Consul v1.9.X series, Envoy v1.11.X will no longer be supported. Current
versions of Nomad hardcode a version of Envoy v1.11.2 to be used as the
default implementation of Connect sidecar proxy.

This PR introduces a change such that each Nomad Client will query its
local Consul for a list of Envoy proxies that it supports (https://github.com/hashicorp/consul/pull/8545)
and then launch the Connect sidecar proxy task using the latest supported version
of Envoy. If the `SupportedProxies` API component is not available from
Consul, Nomad will fallback to the old version of Envoy supported by old
versions of Consul.

Setting the meta configuration option `meta.connect.sidecar_image` or
setting the `connect.sidecar_task` stanza will take precedence as is
the current behavior for sidecar proxies.

Setting the meta configuration option `meta.connect.gateway_image`
will take precedence as is the current behavior for connect gateways.

`meta.connect.sidecar_image` and `meta.connect.gateway_image` may make
use of the special `${NOMAD_envoy_version}` variable interpolation, which
resolves to the newest version of Envoy supported by the Consul agent.

Addresses #8585 #7665
2020-10-13 09:14:12 -05:00
Yoan Blanc 891accb89a
use allow/deny instead of the colored alternatives (#9019)
Signed-off-by: Yoan Blanc <yoan@dosimple.ch>
2020-10-12 08:47:05 -04:00
Tim Gross ecec432653
csi: allow for volume detach to work with gc'd nodes (#9057)
When we try to prefix match the `nomad volume detach` node ID argument, the
node may have been already GC'd. The volume unpublish workflow gracefully
handles this case so that we can free the claim. So make a best effort to find
a node ID among the volume's claimed allocations, or otherwise just use the
node ID we've been given by the user as-is.
2020-10-09 09:45:03 -04:00
Tim Gross 82749bd6a6
csi: allow volume detach to take a node ID prefix (#9041)
Fixes a bug where the `nomad volume detach` command would not accept a node ID
prefix instead of a full node ID. The volume ID is already prefix matched
server-side.
2020-10-07 11:14:57 -04:00
James Rasell d2fe895216
Merge pull request #9023 from hashicorp/f-gh-8648
cli: add scale and scaling-events commands to job cmd.
2020-10-06 18:03:41 +02:00
Dave May 561637c063
Merge pull request #9034 from hashicorp/dmay-debug-metrics
Add metrics command / output to debug bundle
2020-10-06 11:47:09 -04:00
Seth Hoenig 6cffbecb3a
Merge pull request #9033 from pierreca/verify-remove-checks
Do not double-remove checks removed by Consul
2020-10-06 10:16:13 -05:00
davemay99 18aa30c00f metrics return bytes instead of string for more flexibility 2020-10-06 10:49:15 -04:00
davemay99 19a075cf47 update deprecated syntax per GH-9027 2020-10-06 09:47:16 -04:00
davemay99 7160c26f04 sync vendored modules 2020-10-06 09:16:52 -04:00
James Rasell b7dac9020f
Merge pull request #9025 from hashicorp/f-gh-8649
cli: add policy list and info to new scaling cmd.
2020-10-06 12:40:43 +02:00
James Rasell 552d1b2ed4
cli: ensure scaling policy target doesn't have trailing comma 2020-10-06 12:18:17 +02:00
James Rasell 564adc1678
cli: add scale and scaling-events commands to job cmd.
This adds the ability to scale Nomad jobs and view scaling events
via the CLI.
2020-10-06 09:58:46 +02:00
James Rasell ffe6533ad1
Merge pull request #9027 from hashicorp/f-gh-9026
cli: move tests to use NewMockUi func.
2020-10-06 08:28:18 +02:00
davemay99 603cc1776c Add metrics command / output to debug bundle 2020-10-05 22:30:01 -04:00
Pierre Cauchois 1efe05f516 Do not double-remove checks removed by Consul
When deregistering a service, consul also deregisters the associated
checks. The current state keeps track of all services and all checks
separately and deregisters them in sequence, which leads, whether during
syncs or shutdowns, to check deregistrations happening twice and failing
the second time (generating errors in logs)

This fix includes:
- a fix to the sync logic that just pulls the checks *after* the
services have been synced
- a fix to the shutdown mechanism that gets an updated list of checks
after deregistering the services, so that we get a cleaner check
deregistration process.
2020-10-06 00:30:29 +00:00
Chris Baker 7f701fddd0 updated docs and validation to further prohibit null chars in region, datacenter, and job name 2020-10-05 18:01:50 +00:00
James Rasell 2ed78b8a7e
cli: move tests to use NewMockUi func. 2020-10-05 16:07:41 +02:00
James Rasell b8727997cd
cli: add policy list and info to new scaling cmd.
This adds the ability to detail scaling policies via the CLI.
2020-10-05 15:18:30 +02:00
Kent 'picat' Gruber 5e1c716835
Merge pull request #8998 from hashicorp/keygen-32-bytes
Use 32-byte key for gossip encryption to enable AES-256
2020-10-02 17:17:55 -04:00
Kent 'picat' Gruber b03f79700c Fix panic in test due to the agent's logger not being initialized yet
So a null logger is used to avoid the problem.
2020-10-02 11:10:27 -04:00
Fredrik Hoem Grelland 953d4de8dd
update consul-template to v0.25.1 (#8988) 2020-10-01 14:08:49 -04:00
Kent 'picat' Gruber 90e85f9add Fix other usages of initKeyring func to use logger as third argument 2020-10-01 11:13:06 -04:00
Kent 'picat' Gruber b98bb99dfe Log AES-128 and AES-192 key sizes during keyring initialization 2020-10-01 11:12:14 -04:00
Kent 'picat' Gruber 42bdb03f43 Fix operator keygen test to check for 32 bytes 2020-09-30 17:04:33 -04:00
Kent 'picat' Gruber 6cefe03359 Generate 32-byte gossip key for nomad operator keygen command
The key generated from this command is used for gossip encrpytion, which utilizes AES GCM encryption. Using a key size of 16-bytes enables AES-128 while a key size of 32 bytes enables AES-256.

The underlying memberlist library supports the larger key size, and is ultimatley preferable from a security standpoint. Consul also uses 32 bytes by default: 1a14b94441
2020-09-30 17:02:37 -04:00
Michael Schurter 765473e8b0 jobspec: lower min cpu resources from 10->1
Since CPU resources are usually a soft limit it is desirable to allow
setting it as low as possible to allow tasks to run only in "idle" time.

Setting it to 0 is still not allowed to avoid potential unintentional
side effects with allowing a zero value. While there may not be any side
effects this commit attempts to minimize risk by avoiding the issue.

This does *not* change the defaults.
2020-09-30 12:15:13 -07:00
Dave May eaa4f6faf5
Merge pull request #8922 from hashicorp/dmay-raftutil-path
Raftutil cleanup, plus helper function to find raft.db
2020-09-29 15:12:32 -04:00
Tim Gross b12938a9fb
command: fix a typo in the help text for namespaces (#8975) 2020-09-28 12:23:25 -04:00
davemay99 f2b3536da2 refactor functions to find raft.db 2020-09-24 19:00:53 -04:00
Nick Ethier e75a3f349b
command: remove mbits from quota hcl (#8740) 2020-09-24 11:44:59 -04:00
davemay99 bc9fb2a6ee remove extra debug output 2020-09-17 21:42:53 -04:00
davemay99 5a159f1108 Raftutil cleanup, plus helper function to find raft.db 2020-09-17 21:35:17 -04:00
Mahmood Ali 87b0437e0f
Merge pull request #8911 from hashicorp/f-task_network_warning-smaller
Smaller 0.12 mbit deprecation PR
2020-09-17 08:11:13 -05:00
Tim Gross 7a691d0000
filter volumes by type in 'nomad node status' output (#8902)
Volume requests can be either CSI or host volumes, so when displaying the CSI
volume info for `nomad node status -verbose` we need to filter out the host
volumes.
2020-09-16 15:00:12 -04:00
Mahmood Ali d65cda5e70 Update job examples with MBit deprecation 2020-09-16 11:06:19 -04:00
Charlie Voiselle 5ec3945531
Change tabs to spaces in nomad monitor help text 2020-09-14 15:08:30 -04:00
Michael Schurter 1544341f09
Merge pull request #8862 from hashicorp/release-0.12.4
Prepare for 0.13 development cycle
2020-09-10 09:14:44 -07:00
Mahmood Ali d4f385d6e1
Upgrade to golang 1.15 (#8858)
Upgrade to golang 1.15

Starting with golang 1.5, setting Ctty value result in `Setctty set but Ctty not valid in child` error, as part of https://github.com/golang/go/issues/29458 .
This commit lifts the fix in https://github.com/creack/pty/pull/97 .
2020-09-09 15:59:29 -04:00
Nomad Release bot 3b8a2f22dc Generate files for 0.12.4-rc1 release 2020-09-03 02:59:23 +00:00
Drew Bailey 33dc50dca0
Merge pull request #8793 from hashicorp/debug-cli/run-intervals
run commands for duration and interval without needing to specify server or node
2020-08-31 16:07:26 -04:00