docs: update docs for linux capabilities in exec/java/docker drivers
Update docs for allow_caps, cap_add, cap_drop in exec/java/docker driver pages. Also update upgrade guide with guidance on new default linux capabilities for exec and java drivers.
This commit is contained in:
Seth Hoenig
parent
87c96eed11
commit
f64baec276
|
|
@ -23,6 +23,7 @@ type Set struct {
|
|||
data map[string]nothing
|
||||
}
|
||||
|
||||
// New creates a new Set setting caps as the initial elements.
|
||||
func New(caps []string) *Set {
|
||||
m := make(map[string]nothing, len(caps))
|
||||
for _, c := range caps {
|
||||
|
|
|
|||
|
|
@ -452,30 +452,26 @@ config {
|
|||
- `cap_add` - (Optional) A list of Linux capabilities as strings to pass directly to
|
||||
[`--cap-add`](https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities).
|
||||
Effective capabilities (computed from `cap_add` and `cap_drop`) have to match the configured allowlist.
|
||||
The allowlist can be customized using the [`allow_caps`](#plugin_caps) plugin option key in the client node's configuration.
|
||||
The allowlist can be customized using the [`allow_caps`][allow_caps] plugin option key in the client node's configuration.
|
||||
For example:
|
||||
|
||||
```hcl
|
||||
config {
|
||||
cap_add = [
|
||||
"SYS_TIME",
|
||||
]
|
||||
}
|
||||
```
|
||||
```hcl
|
||||
config {
|
||||
cap_add = ["net_raw", sys_time"]
|
||||
}
|
||||
```
|
||||
|
||||
- `cap_drop` - (Optional) A list of Linux capabilities as strings to pass directly to
|
||||
[`--cap-drop`](https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities).
|
||||
Effective capabilities (computed from `cap_add` and `cap_drop`) have to match the configured allowlist.
|
||||
The allowlist can be customized using the [`allow_caps`](#plugin_caps) plugin option key in the client node's configuration.
|
||||
The allowlist can be customized using the [`allow_caps`][allow_caps] plugin option key in the client node's configuration.
|
||||
For example:
|
||||
|
||||
```hcl
|
||||
config {
|
||||
cap_drop = [
|
||||
"MKNOD",
|
||||
]
|
||||
}
|
||||
```
|
||||
```hcl
|
||||
config {
|
||||
cap_drop = ["mknod"]
|
||||
}
|
||||
```
|
||||
|
||||
- `cpu_hard_limit` - (Optional) `true` or `false` (default). Use hard CPU
|
||||
limiting instead of soft limiting. By default this is `false` which means
|
||||
|
|
@ -797,10 +793,7 @@ plugin "docker" {
|
|||
}
|
||||
|
||||
allow_privileged = false
|
||||
allow_caps = ["CHOWN", "NET_RAW"]
|
||||
|
||||
# allow_caps can also be set to "ALL"
|
||||
# allow_caps = ["ALL"]
|
||||
allow_caps = ["chown", "net_raw"]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
|
@ -823,13 +816,22 @@ plugin "docker" {
|
|||
from the Docker engine during an image pull within this timeframe, Nomad will
|
||||
timeout the request that initiated the pull command. (Minimum of `1m`)
|
||||
|
||||
- `allow_caps`<a id="plugin_caps"></a> - A list of allowed Linux capabilities.
|
||||
Defaults to
|
||||
`CHOWN,DAC_OVERRIDE,FSETID,FOWNER,MKNOD,NET_RAW,SETGID,SETUID,SETFCAP,SETPCAP, NET_BIND_SERVICE,SYS_CHROOT,KILL,AUDIT_WRITE` which is the list of
|
||||
capabilities allowed by docker by default, as defined here. Allows the
|
||||
operator to control which capabilities can be obtained by tasks using cap_add
|
||||
and cap_drop options. Supports the value "ALL" as a shortcut for allowlisting
|
||||
all capabilities.
|
||||
- `allow_caps` - A list of allowed Linux capabilities. Defaults to
|
||||
|
||||
```hcl
|
||||
["audit_write", "chown", "dac_override", "fowner", "fsetid", "kill", "mknod",
|
||||
"net_bind_service", "setfcap", "setgid", "setpcap", "setuid", "sys_chroot"]
|
||||
```
|
||||
|
||||
which is the same list of capabilities allowed by [docker by default][docker_caps]
|
||||
(sans [`NET_RAW`][no_net_raw]). Allows the operator to control which capabilities can be obtained
|
||||
by tasks using [`cap_add`][cap_add] and [`cap_drop`][cap_drop] options. Supports
|
||||
the value `"all"` as a shortcut for allow-listing all capabilities supported by
|
||||
the operating system.
|
||||
|
||||
!> **Warning:** Allowing more capabilities beyond the default may lead to
|
||||
undesirable consequences, including untrusted tasks being able to compromise the
|
||||
host system.
|
||||
|
||||
- `allow_runtimes` - defaults to `["runc", "nvidia"]` - A list of the allowed
|
||||
docker runtimes a task may use.
|
||||
|
|
@ -1136,3 +1138,8 @@ Windows is relatively new and rapidly evolving you may want to consult the
|
|||
[plugin-stanza]: /docs/configuration/plugin
|
||||
[allocation working directory]: /docs/runtime/environment#task-directories 'Task Directories'
|
||||
[`auth_soft_fail=true`]: #auth_soft_fail
|
||||
[cap_add]: /docs/drivers/docker#cap_add
|
||||
[cap_drop]: /docs/drivers/docker#cap_drop
|
||||
[no_net_raw]: /docs/upgrade/upgrade-specific#nomad-1-1-0-rc1-1-0-5-0-12-12
|
||||
[docker_caps]: https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities
|
||||
[allow_caps]: /docs/drivers/docker#allow_caps
|
||||
|
|
|
|||
|
|
@ -54,6 +54,27 @@ be able to access sensitive process information like environment variables.
|
|||
!> **Warning:** If set to `"host"`, other processes running as the same user will be
|
||||
able to make use of IPC features, like sending unexpected POSIX signals.
|
||||
|
||||
- `cap_add` - (Optional) A list of Linux capabilities to enable for the task.
|
||||
Effective capabilities (computed from `cap_add` and `cap_drop`) must be a subset
|
||||
of the allowed capabilities configured with [`allow_caps`][allow_caps].
|
||||
|
||||
```hcl
|
||||
config {
|
||||
cap_add = ["net_raw", "sys_time"]
|
||||
}
|
||||
```
|
||||
|
||||
- `cap_drop` - (Optional) A list of Linux capabilities to disable for the task.
|
||||
Effective capabilities (computed from `cap_add` and `cap_drop`) must be a subset
|
||||
of the allowed capabilities configured with [`allow_caps`][allow_caps].
|
||||
|
||||
```hcl
|
||||
config {
|
||||
cap_drop = ["all"]
|
||||
cap_add = ["chown", "sys_chroot", "mknod"]
|
||||
}
|
||||
```
|
||||
|
||||
## Examples
|
||||
|
||||
To run a binary present on the Node:
|
||||
|
|
@ -138,6 +159,23 @@ able to make use of IPC features, like sending unexpected POSIX signals.
|
|||
for file system isolation without `pivot_root`. This is useful for systems
|
||||
where the root is on a ramdisk.
|
||||
|
||||
- `allow_caps` - A list of allowed Linux capabilities. Defaults to
|
||||
|
||||
```hcl
|
||||
["audit_write", "chown", "dac_override", "fowner", "fsetid", "kill", "mknod",
|
||||
"net_bind_service", "setfcap", "setgid", "setpcap", "setuid", "sys_chroot"]
|
||||
```
|
||||
|
||||
which is modeled after the capabilities allowed by [docker by default][docker_caps]
|
||||
(sans [`NET_RAW`][no_net_raw]). Allows the operator to control which capabilities
|
||||
can be obtained by tasks using [`cap_add`][cap_add] and [`cap_drop`][cap_drop] options.
|
||||
Supports the value `"all"` as a shortcut for allow-listing all capabilities supported
|
||||
by the operating system.
|
||||
|
||||
!> **Warning:** Allowing more capabilities beyond the default may lead to
|
||||
undesirable consequences, including untrusted tasks being able to compromise the
|
||||
host system.
|
||||
|
||||
## Client Attributes
|
||||
|
||||
The `exec` driver will set the following client attributes:
|
||||
|
|
@ -200,3 +238,8 @@ This list is configurable through the agent client
|
|||
|
||||
[default_pid_mode]: /docs/drivers/exec#default_pid_mode
|
||||
[default_ipc_mode]: /docs/drivers/exec#default_ipc_mode
|
||||
[cap_add]: /docs/drivers/exec#cap_add
|
||||
[cap_drop]: /docs/drivers/exec#cap_drop
|
||||
[no_net_raw]: /docs/upgrade/upgrade-specific#nomad-1-1-0-rc1-1-0-5-0-12-12
|
||||
[allow_caps]: /docs/drivers/exec#allow_caps
|
||||
[docker_caps]: https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities
|
||||
|
|
|
|||
|
|
@ -61,6 +61,27 @@ be able to access sensitive process information like environment variables.
|
|||
!> **Warning:** If set to `"host"`, other processes running as the same user will be
|
||||
able to make use of IPC features, like sending unexpected POSIX signals.
|
||||
|
||||
- `cap_add` - (Optional) A list of Linux capabilities to enable for the task.
|
||||
Effective capabilities (computed from `cap_add` and `cap_drop`) must be a subset
|
||||
of the allowed capabilities configured with [`allow_caps`][allow_caps].
|
||||
|
||||
```hcl
|
||||
config {
|
||||
cap_add = ["net_raw", "sys_time"]
|
||||
}
|
||||
```
|
||||
|
||||
- `cap_drop` - (Optional) A list of Linux capabilities to disable for the task.
|
||||
Effective capabilities (computed from `cap_add` and `cap_drop`) must be a subset
|
||||
of the allowed capabilities configured with [`allow_caps`][allow_caps].
|
||||
|
||||
```hcl
|
||||
config {
|
||||
cap_drop = ["all"]
|
||||
cap_add = ["chown", "sys_chroot", "mknod"]
|
||||
}
|
||||
```
|
||||
|
||||
## Examples
|
||||
|
||||
A simple config block to run a Java Jar:
|
||||
|
|
@ -138,6 +159,23 @@ be able to access sensitive process information like environment variables.
|
|||
!> **Warning:** If set to `"host"`, other processes running as the same user will be
|
||||
able to make use of IPC features, like sending unexpected POSIX signals.
|
||||
|
||||
- `allow_caps` - A list of allowed Linux capabilities. Defaults to
|
||||
|
||||
```hcl
|
||||
["audit_write", "chown", "dac_override", "fowner", "fsetid", "kill", "mknod",
|
||||
"net_bind_service", "setfcap", "setgid", "setpcap", "setuid", "sys_chroot"]
|
||||
```
|
||||
|
||||
which is modeled after the capabilities allowed by [docker by default][docker_caps]
|
||||
(sans [`NET_RAW`][no_net_raw]). Allows the operator to control which capabilities
|
||||
can be obtained by tasks using [`cap_add`][cap_add] and [`cap_drop`][cap_drop] options.
|
||||
Supports the value `"all"` as a shortcut for allow-listing all capabilities supported
|
||||
by the operating system.
|
||||
|
||||
!> **Warning:** Allowing more capabilities beyond the default may lead to
|
||||
undesirable consequences, including untrusted tasks being able to compromise the
|
||||
host system.
|
||||
|
||||
## Client Requirements
|
||||
|
||||
The `java` driver requires Java to be installed and in your system's `$PATH`. On
|
||||
|
|
@ -208,3 +246,8 @@ This list is configurable through the agent client
|
|||
|
||||
[default_pid_mode]: /docs/drivers/java#default_pid_mode
|
||||
[default_ipc_mode]: /docs/drivers/java#default_ipc_mode
|
||||
[cap_add]: /docs/drivers/java#cap_add
|
||||
[cap_drop]: /docs/drivers/java#cap_drop
|
||||
[no_net_raw]: /docs/upgrade/upgrade-specific#nomad-1-1-0-rc1-1-0-5-0-12-12
|
||||
[allow_caps]: /docs/drivers/java#allow_caps
|
||||
[docker_caps]: https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities
|
||||
|
|
|
|||
|
|
@ -54,7 +54,36 @@ these fields.
|
|||
|
||||
Connect native tasks running in host networking mode will now have `CONSUL_HTTP_ADDR`
|
||||
set automatically. Before this was only the case for bridge networking. If an operator
|
||||
already explicitly set `CONSUL_HTTP_ADDR` then it will not get overriden.
|
||||
already explicitly set `CONSUL_HTTP_ADDR` then it will not get overridden.
|
||||
|
||||
#### Linux capabilities in exec/java
|
||||
|
||||
Following the security [remediation][no_net_raw] in Nomad versions 0.12.12, 1.0.5,
|
||||
and 1.1.0-rc1, the `exec` and `java` task drivers will additionally no longer enable
|
||||
the following linux capabilities by default:
|
||||
|
||||
```
|
||||
AUDIT_CONTROL AUDIT_READ BLOCK_SUSPEND DAC_READ_SEARCH IPC_LOCK IPC_OWNER LEASE
|
||||
LINUX_IMMUTABLE MAC_ADMIN MAC_OVERRIDE NET_ADMIN NET_BROADCAST SYS_ADMIN
|
||||
SYS_BOOT SYSLOG SYS_MODULE SYS_NICE SYS_PACCT SYS_PTRACE SYS_RAWIO SYS_RESOURCE
|
||||
SYS_TIME SYS_TTY_CONFIG WAKE_ALARM
|
||||
```
|
||||
|
||||
The capabilities now enabled by default are modeled after Docker default [`linux capabilities`]:
|
||||
|
||||
```
|
||||
AUDIT_WRITE CHOWN DAC_OVERRIDE FOWNER FSETID KILL MKNOD NET_BIND_SERVICE
|
||||
NET_RAW SETFCAP SETGID SETPCAP SETUID SYS_CHROOT
|
||||
```
|
||||
|
||||
A new `allow_caps` plugin configuration parameter for [`exec`][allow_caps_exec]
|
||||
and [`java`][allow_caps_java] task drivers can be used to restrict the set of
|
||||
capabilities allowed for use by tasks.
|
||||
|
||||
Tasks using the `exec` or `java` task drivers can add or remove desired linux
|
||||
capabilities using the [`cap_add`][cap_add_exec] and [`cap_drop`][cap_drop_exec]
|
||||
task configuration options.
|
||||
|
||||
|
||||
#### iptables
|
||||
|
||||
|
|
@ -63,9 +92,9 @@ inserting them as the first rule. This allows better control for user-defined
|
|||
iptables rules but users who append rules currently should verify that their
|
||||
rules are being appended in the correct order.
|
||||
|
||||
## Nomad 1.1.0, 1.0.5, 0.12.12
|
||||
## Nomad 1.1.0-rc1, 1.0.5, 0.12.12
|
||||
|
||||
Nomad versions 1.1.0, 1.0.5 and 0.12.12 change the behavior of the `docker`, `exec`,
|
||||
Nomad versions 1.1.0-rc1, 1.0.5 and 0.12.12 change the behavior of the `docker`, `exec`,
|
||||
and `java` task drivers so that the [`CAP_NET_RAW`] linux capability is disabled
|
||||
by default. This is one of the [`linux capabilities`] that Docker itself enables
|
||||
by default, as this capability enables the generation of ICMP packets - used by
|
||||
|
|
@ -1111,3 +1140,8 @@ deleted and then Nomad 0.3.0 can be launched.
|
|||
[`CAP_NET_RAW`]: https://security.stackexchange.com/a/128988
|
||||
[`linux capabilities`]: https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities
|
||||
[`allow_caps`]: /docs/drivers/docker#allow_caps
|
||||
[no_net_raw]: /docs/upgrade/upgrade-specific#nomad-1-1-0-rc1-1-0-5-0-12-12
|
||||
[allow_caps_exec]: /docs/drivers/exec#allow_caps
|
||||
[allow_caps_java]: /docs/drivers/java#allow_caps
|
||||
[cap_add_exec]: /docs/drivers/exec#cap_add
|
||||
[cap_drop_exec]: /docs/drivers/exec#cap_drop
|
||||
|
|
|
|||
Loading…
Reference in New Issue