Commit graph

439 commits

Author SHA1 Message Date
Lang Martin 1e33da5fd1 executor_universal_linux log a link to the docs on cgroup error 2019-07-24 12:37:33 -04:00
Lang Martin a1d496c05c executor_universal_linux raw_exec cgroup failure is not fatal 2019-07-22 15:16:36 -04:00
Lang Martin a0fe1ffdd5 default e.getAllPids in executor_basic 2019-07-18 10:57:27 -04:00
Lang Martin 9d0c0c459d executor_unix and _windows stub getAllPids ByScanning 2019-07-17 17:34:06 -04:00
Lang Martin e071f6b022 executor_universal_linux getAllPids chooses cgroup when available 2019-07-17 17:33:55 -04:00
Lang Martin e1bab541ad executor use e.getAllPids() 2019-07-17 17:33:11 -04:00
Lang Martin 18597c4917 resource_container_linux new getAllPidsByCgroup 2019-07-17 17:31:36 -04:00
Lang Martin 2e981a812e pid_collector getAllPids -> getAllPidsByScanning 2019-07-17 17:31:20 -04:00
Jasmine Dahilig 1c1e81b294
Merge pull request #5846 from hashicorp/f-docker-log-constraints
add log rotation to docker driver log defaults
2019-07-03 10:17:19 -07:00
Jasmine Dahilig cece83dd9c default to json-file log rotation for docker driver 2019-07-03 09:04:45 -07:00
Mahmood Ali 6c245c9b6a
Merge pull request #5811 from cloudbuy/b-win32-volume-split
lift code from docker/volume/mounts for splitting windows volumes
2019-06-18 21:19:15 -04:00
Mahmood Ali ac64509c59 comment on use of init() for plugin handlers 2019-06-18 20:54:55 -04:00
Damien Churchill 9ee17f32c0 run new file through goimports 2019-06-18 08:35:25 +01:00
Damien Churchill dba5bd96cd run gofmt over the new file 2019-06-18 08:35:25 +01:00
Damien Churchill 4d7d352d7e drivers/docker: move lifted code out to separate file and link the source & license 2019-06-18 08:35:25 +01:00
Damien Churchill 70daca3395 lift code from docker/volume/mounts for splitting windows volumes
Using the API as provided from the `mounts` package imposes validation
on the `src:dest` which shouldn't be performed at this time. To workaround
that lift the internal code from that library required to only perform
the split.
2019-06-18 08:35:25 +01:00
Mahmood Ali 962921f86c Use init to handle plugin invocation
Currently, nomad "plugin" processes (e.g. executor, logmon, docker_logger) are started as CLI
commands to be handled by command CLI framework.  Plugin launchers use
`discover.NomadBinary()` to identify the binary and start it.

This has few downsides: The trivial one is that when running tests, one
must re-compile the nomad binary as the tests need to invoke the nomad
executable to start plugin.  This is frequently overlooked, resulting in
puzzlement.

The more significant issue with `executor` in particular is in relation
to external driver:

* Plugin must identify the path of invoking nomad binary, which is not
trivial; `discvoer.NomadBinary()` now returns the path to the plugin
rather than to nomad, preventing external drivers from launching
executors.

* The external driver may get a different version of executor than it
expects (specially if we make a binary incompatible change in future).

This commit addresses both downside by having the plugin invocation
handling through an `init()` call, similar to how libcontainer init
handler is done in [1] and recommened by libcontainer [2].  `init()`
will be invoked and handled properly in tests and external drivers.

For external drivers, this change will cause external drivers to launch
the executor that's compiled against.

There a are a couple of downsides to this approach:
* These specific packages (i.e executor, logmon, and dockerlog) need to
be careful in use of `init()`, package initializers.  Must avoid having
command execution rely on any other init in the package.  I prefixed
files with `z_` (golang processes files in lexical order), but ensured
we don't depend on order.
* The command handling is spread in multiple packages making it a bit
less obvious how plugin starts are handled.

[1] drivers/shared/executor/libcontainer_nsenter_linux.go
[2] eb4aeed24f/libcontainer (using-libcontainer)
2019-06-13 16:48:01 -04:00
Mahmood Ali 5734c8a648 update comment 2019-06-11 13:00:26 -04:00
Mahmood Ali f7608c4cef exec: use an independent name=systemd cgroup path
We aim for containers to be part of a new cgroups hierarchy independent
from nomad agent.  However, we've been setting a relative path as
libcontainer `cfg.Cgroups.Path`, which makes libcontainer concatinate
the executor process cgroup with passed cgroup, as set in [1].

By setting an absolute path, we ensure that all cgroups subsystem
(including `name=systemd` get a dedicated one).  This matches behavior
in Nomad 0.8, and behavior of how Docker and OCI sets CgroupsPath[2]

Fixes #5736

[1] d7edf9b2e4/vendor/github.com/opencontainers/runc/libcontainer/cgroups/fs/apply_raw.go (L326-L340)
[2] 238f8eaa31/vendor/github.com/containerd/containerd/oci/spec.go (L229)
2019-06-10 22:00:12 -04:00
Chris Baker 3ca97d52db docker/driver: downgraded log level for error in DestroyTask 2019-06-03 21:21:32 +00:00
Chris Baker 2af897c76f drivers/docker: modify container/image cleanup to be robust to containers removed out of band 2019-06-03 19:52:28 +00:00
Chris Baker be6c6e8ce1 docker/tests:
- modified tests to cleanup now that RemoveContainer isn't in StartTask
- fix some broken tests by removing docker images/containers before test
2019-06-03 19:05:08 +00:00
Chris Baker 9442c26cff docker: DestroyTask was not cleaning up Docker images because it was erroring early due to an attempt to inspect an image that had already been removed 2019-06-03 19:04:27 +00:00
Mahmood Ali cb554a015f Fix test comparisons 2019-05-24 21:38:22 -05:00
Mahmood Ali 99637c8bbc Test for expected capabilities specifically 2019-05-24 16:07:05 -05:00
Mahmood Ali 7455c746aa use /bin/bash 2019-05-24 14:50:23 -04:00
Mahmood Ali 68813def56 special case root capabilities 2019-05-24 14:10:10 -04:00
Mahmood Ali 01d5c90cbb tests: Fix binary dir permissions 2019-05-24 11:31:12 -04:00
Mahmood Ali 00081b15d6 fix 2019-05-20 15:30:07 -04:00
Mahmood Ali 807e7b90e0 drivers/exec: Restore 0.8 capabilities
Nomad 0.9 incidentally set effective capabilities that is higher than
what's expected of a `nobody` process, and what's set in 0.8.

This change restores the capabilities to ones used in Nomad 0.9.
2019-05-20 13:11:29 -04:00
Lang Martin 0256cf700d
Merge pull request #5649 from hashicorp/b-lookup-exe-chroot
lookup executables inside chroot
2019-05-17 15:07:41 -04:00
Michael Schurter 8589233a0e drivers/mock: implement InspectTask 2019-05-14 10:53:27 -07:00
Mahmood Ali dd8762e348 typo: "atleast" -> "at least" 2019-05-13 10:01:19 -04:00
Mahmood Ali d1526571a5 implement nomad exec for rkt
Implement the streaming exec handler for the rkt driver
2019-05-12 18:59:00 -04:00
Mahmood Ali b4df061fef use pty/tty terminology similar to github.com/kr/pty 2019-05-10 19:17:14 -04:00
Mahmood Ali a4640db7a6 drivers: implement streaming exec for executor based drivers
These simply delegate call to backend executor.
2019-05-10 19:17:14 -04:00
Mahmood Ali 3055fd53df executors: implement streaming exec
Implements streamign exec handling in both executors (i.e. universal and
libcontainer).

For creation of TTY, some incidental complexity leaked in.  The universal
executor uses github.com/kr/pty for creation of TTYs.

On the other hand, libcontainer expects a console socket and for libcontainer to
create the underlying console object on process start.  The caller can then use
`libcontainer.utils.RecvFd()` to get tty master end.

I chose github.com/kr/pty for managing TTYs here.  I tried
`github.com/containerd/console` package (which is already imported), but the
package did not work as expected on macOS.
2019-05-10 19:17:14 -04:00
Mahmood Ali 085d2ef759 executor: scaffolding for executor grpc handling
Prepare executor to handle streaming exec API calls that reuse drivers protobuf
structs.
2019-05-10 19:17:14 -04:00
Lang Martin 99359d7fbe executor_linux only do path resolution in the taskDir, not local
split out lookPathIn to show it's similarity to exec.LookPath
2019-05-10 11:33:35 -04:00
Mahmood Ali 13c83ee38e drivers/docker: implement streaming exec 2019-05-09 16:49:08 -04:00
Mahmood Ali ce97689b79 drivers/mock: implement nomad exec interface 2019-05-09 16:49:08 -04:00
Lang Martin 3ae276cfd2 executor_linux_test call lookupTaskBin with an ExecCommand 2019-05-08 10:01:51 -04:00
Lang Martin 743a2a2875 executor_linux pass the command to lookupTaskBin to get path 2019-05-08 10:01:20 -04:00
Lang Martin 8db3fe047c executor/* Launch log at top of Launch is more explicit, trace 2019-05-07 17:01:05 -04:00
Lang Martin 87585e950d move lookupTaskBin to executor_linux, for os dependency clarity 2019-05-07 16:58:27 -04:00
Lang Martin de807a410a driver_test leave cat in the test, but add cat to the chroot 2019-05-07 16:14:01 -04:00
Lang Martin 1e5d851d23 executor_test cleanup old lookupBin tests 2019-05-04 10:21:59 -04:00
Lang Martin c777271374 driver_test StartWait task calls a program inside it's chroot 2019-05-03 16:34:31 -04:00
Lang Martin c0741e392d executor lookupTaskBin also does PATH expansion, anchored in taskDIR 2019-05-03 16:22:09 -04:00
Lang Martin 1619d3e3cb executor_linux_test test PATH lookup inside the container 2019-05-03 16:21:58 -04:00