Commit graph

77 commits

Author SHA1 Message Date
Danielle Tomlinson 0544a57abe drivers: Move client/drivers/executor to drivers/shared/executor 2018-11-30 10:46:13 +01:00
Nick Ethier d94b631b6b
drivers/exec: add exec implementation 2018-10-16 22:45:28 -04:00
Nick Ethier 5dee1141d1 executor v2 (#4656)
* client/executor: refactor client to remove interpolation

* executor: POC libcontainer based executor

* vendor: use hashicorp libcontainer fork

* vendor: add libcontainer/nsenter dep

* executor: updated executor interface to simplify operations

* executor: implement logging pipe

* logmon: new logmon plugin to manage task logs

* driver/executor: use logmon for log management

* executor: fix tests and windows build

* executor: fix logging key names

* executor: fix test failures

* executor: add config field to toggle between using libcontainer and standard executors

* logmon: use discover utility to discover nomad executable

* executor: only call libcontainer-shim on main in linux

* logmon: use seperate path configs for stdout/stderr fifos

* executor: windows fixes

* executor: created reusable pid stats collection utility that can be used in an executor

* executor: update fifo.Open calls

* executor: fix build

* remove executor from docker driver

* executor: Shutdown func to kill and cleanup executor and its children

* executor: move linux specific universal executor funcs to seperate file

* move logmon initialization to a task runner hook

* client: doc fixes and renaming from code review


* taskrunner: use shared config struct for logmon fifo fields

* taskrunner: logmon only needs to be started once per task
2018-10-16 16:53:31 -07:00
Nick Ethier 66ff12e5f7 Update runc/libcontainer and friends (#4655)
* vendor: bump libcontainer and docker to remove Sirupsen imports

* vendor: fix bad vendoring of archive package

* vendor: fix api changes to cgroups in executor

* vendor: fix docker api changes

* vendor: update github.com/Azure/go-ansiterm to use non capitalized logrus import
2018-10-16 16:53:30 -07:00
Alex Dadgar ff28b04c46 Use more appropriate name than cgroup 2018-05-31 18:05:03 -07:00
Alex Dadgar 37e900b1d3 Only use freezer/devices when in the basic cgroup only 2018-05-31 18:05:03 -07:00
Alex Dadgar ffd9270f2f Use cgroup when possible 2018-05-31 18:05:03 -07:00
Fabian Holler e6af051c93 fix: disable swap for executor_linux allocations
A comment in the nomad source code states that swapping for
executor_linux allocations is disabled but it wasn't.

Nomad wrote -1 to the memsw.limit_in_bytes cgroup file to disable
swapping.
This has the following problems:

1.) Writing -1 to the file does not disable swapping. It sets
    the limit for memory and swap to unlimited.
2.) On common Linux distributions like Ubuntu 16.04 LTS the
    memsw.limit_in_bytes cgroup file does not exist by default.
    The memsw.limit_in_bytes file only exist if the Linux kernel is
    build with CONFIG_MEMCG_SWAP=yes and either
    CONFIG_MEMCG_SWAP_ENABLED=yes or when the kernel parameter
    swapaccount=1 is passed during boot.
    Most Linux distributions disable swap accounting by default because
    of higher memory usage.
    Nomad silently ignores if writing to the memsw.limit_in_bytes file
    fails. The allocation succeeds, no message is logged to notify the
    user.

To ensure that disabling swap works on common Linux kernels, disable
swapping by writing 0 to the memory.swappiness file.
Using the memory.swappiness file only requires that the kernel is
compiled with CONFIG_MEMCG=yes. This is the default in common Linux
kernels.
2018-03-13 10:52:50 +01:00
Michael Schurter a66c53d45a Remove structs import from api
Goes a step further and removes structs import from api's tests as well
by moving GenerateUUID to its own package.
2017-09-29 10:36:08 -07:00
Alex Dadgar a171a014b3 Various fixes for setting user/group of task
This PR fixes two issues:
* Folder permissions in -dev mode were incorrect and not suitable for
running as a particular user.
* Was not setting the group membership properly for the launched
process.

Fixes https://github.com/hashicorp/nomad/issues/2160
2017-03-20 14:21:13 -07:00
Michael Schurter 3ea09ba16a Move chroot building into TaskRunner
* Refactor AllocDir to have a TaskDir struct per task.
* Drivers expose filesystem isolation preference
* Fix lxc mounting of `secrets/`
2017-01-05 16:31:49 -08:00
Alex Dadgar 742e11ddb4 Fix env vars relating to secretdir 2016-11-08 12:28:43 -08:00
Alex Dadgar 2c8dd8bbd3 Revert "Introduce a Secret/ directory" 2016-09-01 17:23:15 -07:00
Alex Dadgar 0626eb9619 environment variables 2016-08-31 13:56:11 -07:00
Jay Oster 2ae059b41d Address review comments
- Simplify map length check in Linux Executor
- Added a `chroot_env` test for config parser
- Moved `ChrootEnv` field from ExecutorCommand to ExecutorContext
- Added a test for `chroot_env` functionality
2016-08-04 15:33:06 -07:00
Jay Oster 7df692226a Add config field to specify chroot mapping for exec driver
- Same format as used by the internal chroot mapping
- Map: source_path -> dest_path
- Example HCL:

client {
  chroot_env {
    "/etc" = "/etc"
    "/lib" = "/lib"
    "/opt/projects/foo/bin" = "/usr/bin"
  }
}
2016-08-03 17:17:17 -07:00
Sean Chittenden be272168c7
Rename resourceContainer{,Context} and resCon{,Ctx}. 2016-07-11 00:02:55 -07:00
Sean Chittenden d4fe69ddf9
Update comments and pushdown a lock into the resource container 2016-07-10 00:12:59 -07:00
Sean Chittenden fc9cd8d4af
Push down the Linux-specific bits into resourceContainer 2016-07-10 00:06:53 -07:00
Alex Dadgar 2628827b7c Merge pull request #1262 from hashicorp/remove-artifact-check
Removing artifact check for java and qemu drivers
2016-06-12 11:21:18 -07:00
Diptanu Choudhury beb362e202 Setting a flag to indicate whether fs isolation is indeed happening 2016-06-12 15:43:24 +02:00
Diptanu Choudhury 641cf50682 Not converting the abs path relative to task dir for drivers which enforce FS isolation only in linux 2016-06-12 13:54:30 +02:00
Alex Dadgar e952540f6f Allocation resources returned in a struct 2016-06-11 21:04:10 -07:00
Diptanu Choudhury f94f89b6d7 Pruning out pids which are no longer present 2016-06-11 01:40:52 +02:00
Diptanu Choudhury 0a9a3918d6 Not reset-ing the list of pids if they don't change 2016-06-11 01:19:50 +02:00
Diptanu Choudhury c38a6fb3c5 Implementing the total ticks per task for the docker driver 2016-06-10 23:33:25 +02:00
Diptanu Choudhury 2d3798b076 Calculating the cpu ticks in nomad client 2016-06-10 22:22:32 +02:00
Alex Dadgar 3cf74e7fd8 Alloc-status only shows measured statistics and fixes to CPU calculations 2016-06-10 10:38:29 -07:00
Diptanu Choudhury 287e0eff22 Fixed merge conflicts 2016-06-10 04:47:34 +01:00
Diptanu Choudhury a69307cff2 Using named variables 2016-06-10 04:45:16 +01:00
Diptanu Choudhury 9056be6d86 Fixed calculation of cpu usage percent in user and sys mode for pids 2016-06-10 04:06:32 +01:00
Alex Dadgar 98d7547014 Annotate what is measured 2016-06-09 19:45:41 -07:00
Diptanu Choudhury 11d1dd113b Added a test for docker 2016-05-28 19:59:20 -07:00
Diptanu Choudhury a0c279f3b2 comments 2016-05-28 19:59:20 -07:00
Diptanu Choudhury a64062d6a6 Fixed the compilation on linux 2016-05-28 19:59:20 -07:00
Diptanu Choudhury 98016ec066 Incorporated review comments for executor 2016-05-28 19:59:20 -07:00
Diptanu Choudhury 313c61b9db Added pidstats in task resource usage struct 2016-05-28 19:59:20 -07:00
Diptanu Choudhury cfd3016747 Renamed CpuUsage to CpuStats 2016-05-28 19:59:20 -07:00
Diptanu Choudhury 193bbc10f8 Fixed the percentage calculation for cgroups 2016-05-28 19:59:20 -07:00
Diptanu Choudhury 22c3f6fee0 Implemented nomad cpu percentage calculator 2016-05-28 19:59:03 -07:00
Diptanu Choudhury e37a108b4c Implemented cpu stats 2016-05-28 19:51:22 -07:00
Diptanu Choudhury c347b27e6a Implemented stats for raw_exec 2016-05-28 19:42:34 -07:00
Diptanu Choudhury 91d2cf319e Added some documentation 2016-05-28 19:42:34 -07:00
Diptanu Choudhury f3d0aecafe Reporting time series of stats 2016-05-28 19:42:34 -07:00
Diptanu Choudhury 13825f434e Added the nomad stats command 2016-05-28 19:42:34 -07:00
Diptanu Choudhury 2b86bb3396 Added cpu stats 2016-05-28 19:42:34 -07:00
Diptanu Choudhury 0fb0e0237f Added a client API to display resource usage of an allocation 2016-05-28 19:42:34 -07:00
Diptanu Choudhury 6c1b60ff07 Add the Stats api to driverhandle 2016-05-28 19:42:34 -07:00
Alex Dadgar ca94d25cb4 Freeze the cgroup and cleanup around Shutdown 2016-04-19 13:48:02 -07:00
Alex Dadgar 736ef76cc1 Put the executor into the cgroup to avoid a fork race 2016-04-18 17:59:08 -07:00