2015-09-15 01:10:11 +00:00
---
2020-02-06 23:45:31 +00:00
layout: docs
page_title: 'Commands: node drain'
sidebar_title: drain
description: |
2018-03-22 17:56:06 +00:00
The node drain command is used to configure a node's drain strategy.
2015-09-15 01:10:11 +00:00
---
2018-03-22 17:56:06 +00:00
# Command: node drain
2015-09-15 01:10:11 +00:00
2018-03-22 17:56:06 +00:00
The `node drain` command is used to toggle drain mode on a given node. Drain
2015-09-15 20:52:02 +00:00
mode prevents any new tasks from being allocated to the node, and begins
2018-03-30 00:52:35 +00:00
migrating all existing allocations away. Allocations will be migrated according
to their [`migrate`][migrate] stanza until the drain's deadline is reached.
By default the `node drain` command blocks until a node is done draining and
2020-02-06 23:45:31 +00:00
all allocations have terminated. Canceling the `node drain` command _will not_
2018-03-30 00:52:35 +00:00
cancel the drain. Drains may be canceled by using the `-disable` parameter
below.
When draining more than one node at a time, it is recommended you first disable
2018-03-30 23:24:19 +00:00
[scheduling eligibility][eligibility] on all nodes that will be drained. For
2020-02-06 23:45:31 +00:00
example if you are decommissioning an entire class of nodes, first run `node eligibility -disable` on all of their node IDs, and then run `node drain -enable`. This will ensure allocations drained from the first node are not
2018-03-30 00:52:35 +00:00
placed on another node about to be drained.
2015-09-15 01:10:11 +00:00
2019-10-22 13:44:00 +00:00
The [node status] command compliments this nicely by providing the current drain
status of a given node.
2015-09-15 03:24:04 +00:00
2019-10-22 13:44:00 +00:00
See the [Workload Migration guide] for detailed examples of node draining.
2018-04-12 22:50:46 +00:00
2015-09-15 01:10:11 +00:00
## Usage
2019-10-22 13:44:00 +00:00
```plaintext
2018-03-22 17:56:06 +00:00
nomad node drain [options] <node>
2015-09-15 01:10:11 +00:00
```
2016-04-11 22:20:49 +00:00
A `-self` flag can be used to drain the local node. If this is not supplied, a
node ID or prefix must be provided. If there is an exact match, the drain mode
will be adjusted for that node. Otherwise, a list of matching nodes and
information will be displayed.
2016-01-06 21:46:57 +00:00
It is also required to pass one of `-enable` or `-disable`, depending on which
operation is desired.
2015-09-15 01:10:11 +00:00
## General Options
2020-02-06 23:45:31 +00:00
@include 'general_options.mdx'
2015-09-15 20:39:57 +00:00
2018-03-22 17:56:06 +00:00
## Drain Options
2015-09-15 20:39:57 +00:00
2019-10-22 13:44:00 +00:00
- `-enable`: Enable node drain mode.
- `-disable`: Disable node drain mode.
- `-deadline`: Set the deadline by which all allocations must be moved off the
2018-03-30 00:52:35 +00:00
node. Remaining allocations after the deadline are force removed from the
node. Defaults to 1 hour.
2019-10-22 13:44:00 +00:00
- `-detach`: Return immediately instead of entering monitor mode.
- `-monitor`: Enter monitor mode directly without modifying the drain status.
- `-force`: Force remove allocations off the node immediately.
- `-no-deadline`: No deadline allows the allocations to drain off the node
2018-03-30 00:52:35 +00:00
without being force stopped after a certain deadline.
2019-10-22 13:44:00 +00:00
- `-ignore-system`: Ignore system allows the drain to complete without stopping
2018-03-30 00:52:35 +00:00
system job allocations. By default system jobs are stopped last.
2019-10-22 13:44:00 +00:00
- `-keep-ineligible`: Keep ineligible will maintain the node's scheduling
2018-03-30 00:52:35 +00:00
ineligibility even if the drain is being disabled. This is useful when an
existing drain is being cancelled but additional scheduling on the node is not
desired.
2019-10-22 13:44:00 +00:00
- `-self`: Drain the local node.
- `-yes`: Automatic yes to prompts.
2015-09-15 03:24:04 +00:00
## Examples
2016-04-11 22:20:49 +00:00
Enable drain mode on node with ID prefix "4d2ba53b":
2015-09-15 03:24:04 +00:00
2020-05-18 20:53:06 +00:00
```shell-session
$ nomad node drain -enable f4e8a9e5
2018-03-30 00:52:35 +00:00
Are you sure you want to enable drain mode for node "f4e8a9e5-30d8-3536-1e6f-cda5c869c35e"? [y/N] y
2018-03-30T23:13:16Z: Ctrl-C to stop monitoring: will not cancel the node drain
2018-03-30T23:13:16Z: Node "f4e8a9e5-30d8-3536-1e6f-cda5c869c35e" drain strategy set
2018-03-30T23:13:17Z: Alloc "1877230b-64d3-a7dd-9c31-dc5ad3c93e9a" marked for migration
2018-03-30T23:13:17Z: Alloc "1877230b-64d3-a7dd-9c31-dc5ad3c93e9a" draining
2018-03-30T23:13:17Z: Alloc "1877230b-64d3-a7dd-9c31-dc5ad3c93e9a" status running -> complete
2018-03-30T23:13:29Z: Alloc "3fce5308-818c-369e-0bb7-f61f0a1be9ed" marked for migration
2018-03-30T23:13:29Z: Alloc "3fce5308-818c-369e-0bb7-f61f0a1be9ed" draining
2018-03-30T23:13:30Z: Alloc "3fce5308-818c-369e-0bb7-f61f0a1be9ed" status running -> complete
2018-03-30T23:13:41Z: Alloc "9a98c5aa-a719-2f34-ecfc-0e6268b5d537" marked for migration
2018-03-30T23:13:41Z: Alloc "9a98c5aa-a719-2f34-ecfc-0e6268b5d537" draining
2018-04-24 18:02:34 +00:00
2018-03-30T23:13:41Z: Node "f4e8a9e5-30d8-3536-1e6f-cda5c869c35e" has marked all allocations for migration
2018-03-30 00:52:35 +00:00
2018-03-30T23:13:42Z: Alloc "9a98c5aa-a719-2f34-ecfc-0e6268b5d537" status running -> complete
2018-03-30T23:13:42Z: All allocations on node "f4e8a9e5-30d8-3536-1e6f-cda5c869c35e" have stopped.
2016-04-11 22:20:49 +00:00
```
Enable drain mode on the local node:
2020-05-18 20:53:06 +00:00
```shell-session
$ nomad node drain -enable -self
2018-03-30 00:52:35 +00:00
...
```
Enable drain mode but do not stop system jobs:
2020-05-18 20:53:06 +00:00
```shell-session
$ nomad node drain -enable -ignore-system 4d2ba53b
2018-03-30 00:52:35 +00:00
...
2015-09-15 03:24:04 +00:00
```
2018-03-30 00:52:35 +00:00
Disable drain mode but keep the node ineligible for scheduling. Useful for
inspecting the current state of a misbehaving node without Nomad trying to
start or migrate allocations:
2020-05-18 20:53:06 +00:00
```shell-session
$ nomad node drain -disable -keep-ineligible 4d2ba53b
2018-03-30 00:52:35 +00:00
...
```
2018-05-10 02:54:05 +00:00
Enable drain mode and detach from monitoring, then reattach later:
2020-05-18 20:53:06 +00:00
```shell-session
$ nomad node drain -enable -detach -self
2018-05-10 02:54:05 +00:00
...
$ nomad node drain -self -monitor
...
```
2018-03-30 00:52:35 +00:00
2020-02-06 23:45:31 +00:00
[eligibility]: /docs/commands/node/eligibility
[migrate]: /docs/job-specification/migrate
[node status]: /docs/commands/node/status
2020-05-03 16:50:40 +00:00
[workload migration guide]: https://learn.hashicorp.com/nomad/operating-nomad/node-draining