Commit graph

467 commits

Author SHA1 Message Date
Mahmood Ali 87f0457973 fix qemu and update docker with tests 2019-09-04 11:27:51 -04:00
Jasmine Dahilig 5b6e39b37c fix portmap envvars in docker driver 2019-09-04 11:26:13 -04:00
Michael Schurter 8fe42fccb0
Merge pull request #6000 from Iqoqo/docker-convert-host-paths-to-host-native
driver/docker: convert host bind path to os native
2019-09-03 09:34:56 -07:00
Danielle Lancashire 724586ba1d
docker: Fix driver spec
hclspec.NewLiteral does not quote its values, which caused `3m` to be
parsed as a nonsensical literal which broke the plugin loader during
initialization. By quoting the value here, it starts correctly.
2019-09-03 08:53:37 +02:00
Zhiguang Wang 832df1091b Add default value "3m" to image_delay, making it consistent with docs. 2019-09-02 16:40:00 +08:00
Mahmood Ali f98d4ee3f1 tests: enable raw_exec driver 2019-08-29 20:26:50 -04:00
Mahmood Ali 28e473aaff raw_exec: be defensive when disabled
Ensure that no raw_exec task can run on a client where it's disabled,
even if a flaw lead to client being assigned a raw_exec task
unexpectedly.
2019-08-29 09:09:40 -04:00
Danielle Lancashire fb63259921
docker: Fix issue where an exec may never timeout 2019-08-16 15:40:03 +02:00
Michael Schurter 83dbac65b2 docker: reword FromSlash(hostPath) comment 2019-08-12 14:38:31 -07:00
ilya guterman 92ce8a0a49 Update utils.go 2019-08-12 19:31:34 +03:00
Ilya Guterman c4b4d7fa43 add comment 2019-08-12 19:31:33 +03:00
Ilya Guterman 52aab40fb3 driver/docker: convert host bind path to os native
relative mounting can be specified using backslashes or forward slashes.
so no prior knowledge of host OS is needed for relative volumes mounting
2019-08-12 19:31:33 +03:00
Michael Schurter aeeec126f5
Merge pull request #5999 from Iqoqo/use-default-network-for-docker
driver/docker: use default network mode
2019-08-01 09:58:12 -07:00
Ilya Guterman a4931ba25b driver/docker: support unix destination mount path in windows
This reverts commit a6c96eade56f0b8880edbec3c4392934492f09bf.
2019-08-01 19:54:08 +03:00
Ilya Guterman 1e6ea0af8c driver/docker: use default network mode
fallback to docker default network mode instead of explicit bridge for linux
or nat for windows
2019-07-31 21:07:46 +03:00
Nick Ethier 1dae42ab81
docker: allow configuration of infra image 2019-07-31 01:04:07 -04:00
Nick Ethier 533b2850fc
executor: cleanup netns handling in executor 2019-07-31 01:04:05 -04:00
Nick Ethier b8a1ebb3b7
executor: support network namespacing on universal executor 2019-07-31 01:03:58 -04:00
Nick Ethier 0e40063092
docker: add nil check on network isolation spec 2019-07-31 01:03:21 -04:00
Nick Ethier f50fa7ef08
docker: fix driver test from changed func args 2019-07-31 01:03:20 -04:00
Nick Ethier d752734719
docker: add additional commens 2019-07-31 01:03:20 -04:00
Nick Ethier 1fc5f86a7c
docker: support shared network namespaces 2019-07-31 01:03:20 -04:00
Nick Ethier 971c8c9c2b
Driver networking support
Adds support for passing network isolation config into drivers and
implements support in the rawexec driver as a proof of concept
2019-07-31 01:03:20 -04:00
Nick Ethier 2d60ef64d9
plugins/driver: make DriverNetworkManager interface optional 2019-07-31 01:03:19 -04:00
Nick Ethier 548f78ef15
ar: initial driver based network management 2019-07-31 01:03:17 -04:00
Michael Schurter ea7fac7bcf
Revert "driver/docker: support unix destination mount path in windows" 2019-07-29 09:09:54 -07:00
Michael Schurter 5bd655e87d
Merge pull request #6001 from Iqoqo/allow-unix-container-path-in-lcow
driver/docker: support unix destination mount path in windows
2019-07-29 09:09:41 -07:00
Lang Martin 1e33da5fd1 executor_universal_linux log a link to the docs on cgroup error 2019-07-24 12:37:33 -04:00
Ilya Guterman cb2284fc3a driver/docker: support unix destination mount path in windows 2019-07-24 11:57:28 +03:00
Lang Martin a1d496c05c executor_universal_linux raw_exec cgroup failure is not fatal 2019-07-22 15:16:36 -04:00
Lang Martin a0fe1ffdd5 default e.getAllPids in executor_basic 2019-07-18 10:57:27 -04:00
Lang Martin 9d0c0c459d executor_unix and _windows stub getAllPids ByScanning 2019-07-17 17:34:06 -04:00
Lang Martin e071f6b022 executor_universal_linux getAllPids chooses cgroup when available 2019-07-17 17:33:55 -04:00
Lang Martin e1bab541ad executor use e.getAllPids() 2019-07-17 17:33:11 -04:00
Lang Martin 18597c4917 resource_container_linux new getAllPidsByCgroup 2019-07-17 17:31:36 -04:00
Lang Martin 2e981a812e pid_collector getAllPids -> getAllPidsByScanning 2019-07-17 17:31:20 -04:00
Jasmine Dahilig 1c1e81b294
Merge pull request #5846 from hashicorp/f-docker-log-constraints
add log rotation to docker driver log defaults
2019-07-03 10:17:19 -07:00
Jasmine Dahilig cece83dd9c default to json-file log rotation for docker driver 2019-07-03 09:04:45 -07:00
Mahmood Ali 6c245c9b6a
Merge pull request #5811 from cloudbuy/b-win32-volume-split
lift code from docker/volume/mounts for splitting windows volumes
2019-06-18 21:19:15 -04:00
Mahmood Ali ac64509c59 comment on use of init() for plugin handlers 2019-06-18 20:54:55 -04:00
Damien Churchill 9ee17f32c0 run new file through goimports 2019-06-18 08:35:25 +01:00
Damien Churchill dba5bd96cd run gofmt over the new file 2019-06-18 08:35:25 +01:00
Damien Churchill 4d7d352d7e drivers/docker: move lifted code out to separate file and link the source & license 2019-06-18 08:35:25 +01:00
Damien Churchill 70daca3395 lift code from docker/volume/mounts for splitting windows volumes
Using the API as provided from the `mounts` package imposes validation
on the `src:dest` which shouldn't be performed at this time. To workaround
that lift the internal code from that library required to only perform
the split.
2019-06-18 08:35:25 +01:00
Mahmood Ali 962921f86c Use init to handle plugin invocation
Currently, nomad "plugin" processes (e.g. executor, logmon, docker_logger) are started as CLI
commands to be handled by command CLI framework.  Plugin launchers use
`discover.NomadBinary()` to identify the binary and start it.

This has few downsides: The trivial one is that when running tests, one
must re-compile the nomad binary as the tests need to invoke the nomad
executable to start plugin.  This is frequently overlooked, resulting in
puzzlement.

The more significant issue with `executor` in particular is in relation
to external driver:

* Plugin must identify the path of invoking nomad binary, which is not
trivial; `discvoer.NomadBinary()` now returns the path to the plugin
rather than to nomad, preventing external drivers from launching
executors.

* The external driver may get a different version of executor than it
expects (specially if we make a binary incompatible change in future).

This commit addresses both downside by having the plugin invocation
handling through an `init()` call, similar to how libcontainer init
handler is done in [1] and recommened by libcontainer [2].  `init()`
will be invoked and handled properly in tests and external drivers.

For external drivers, this change will cause external drivers to launch
the executor that's compiled against.

There a are a couple of downsides to this approach:
* These specific packages (i.e executor, logmon, and dockerlog) need to
be careful in use of `init()`, package initializers.  Must avoid having
command execution rely on any other init in the package.  I prefixed
files with `z_` (golang processes files in lexical order), but ensured
we don't depend on order.
* The command handling is spread in multiple packages making it a bit
less obvious how plugin starts are handled.

[1] drivers/shared/executor/libcontainer_nsenter_linux.go
[2] eb4aeed24f/libcontainer (using-libcontainer)
2019-06-13 16:48:01 -04:00
Mahmood Ali 5734c8a648 update comment 2019-06-11 13:00:26 -04:00
Mahmood Ali f7608c4cef exec: use an independent name=systemd cgroup path
We aim for containers to be part of a new cgroups hierarchy independent
from nomad agent.  However, we've been setting a relative path as
libcontainer `cfg.Cgroups.Path`, which makes libcontainer concatinate
the executor process cgroup with passed cgroup, as set in [1].

By setting an absolute path, we ensure that all cgroups subsystem
(including `name=systemd` get a dedicated one).  This matches behavior
in Nomad 0.8, and behavior of how Docker and OCI sets CgroupsPath[2]

Fixes #5736

[1] d7edf9b2e4/vendor/github.com/opencontainers/runc/libcontainer/cgroups/fs/apply_raw.go (L326-L340)
[2] 238f8eaa31/vendor/github.com/containerd/containerd/oci/spec.go (L229)
2019-06-10 22:00:12 -04:00
Chris Baker 3ca97d52db docker/driver: downgraded log level for error in DestroyTask 2019-06-03 21:21:32 +00:00
Chris Baker 2af897c76f drivers/docker: modify container/image cleanup to be robust to containers removed out of band 2019-06-03 19:52:28 +00:00
Chris Baker be6c6e8ce1 docker/tests:
- modified tests to cleanup now that RemoveContainer isn't in StartTask
- fix some broken tests by removing docker images/containers before test
2019-06-03 19:05:08 +00:00