Commit graph

101 commits

Author SHA1 Message Date
Nick Ethier 5dee1141d1 executor v2 (#4656)
* client/executor: refactor client to remove interpolation

* executor: POC libcontainer based executor

* vendor: use hashicorp libcontainer fork

* vendor: add libcontainer/nsenter dep

* executor: updated executor interface to simplify operations

* executor: implement logging pipe

* logmon: new logmon plugin to manage task logs

* driver/executor: use logmon for log management

* executor: fix tests and windows build

* executor: fix logging key names

* executor: fix test failures

* executor: add config field to toggle between using libcontainer and standard executors

* logmon: use discover utility to discover nomad executable

* executor: only call libcontainer-shim on main in linux

* logmon: use seperate path configs for stdout/stderr fifos

* executor: windows fixes

* executor: created reusable pid stats collection utility that can be used in an executor

* executor: update fifo.Open calls

* executor: fix build

* remove executor from docker driver

* executor: Shutdown func to kill and cleanup executor and its children

* executor: move linux specific universal executor funcs to seperate file

* move logmon initialization to a task runner hook

* client: doc fixes and renaming from code review


* taskrunner: use shared config struct for logmon fifo fields

* taskrunner: logmon only needs to be started once per task
2018-10-16 16:53:31 -07:00
Michael Schurter 7132b67c1e Add Network method to Handle interface
Should probably be moved to an Inspect method in the Driver Plugin world
2018-10-16 16:53:30 -07:00
Javier Palomo Almena 74d3c5df07 DriverContext: Add the TaskGroup and the Job name
Adding this fields to the DriverContext object, will allow us to pass
them to the drivers.

An use case for this, will be to emit tagged metrics in the drivers,
which contain all relevant information:
- Job
- TaskGroup
- Task
- ...

Ref: https://github.com/hashicorp/nomad/pull/4185
2018-04-23 00:15:29 +02:00
Josh Soref e949d23e1b spelling: resource 2018-03-11 18:51:03 +00:00
Josh Soref 2a1cf2f216 spelling: initialization 2018-03-11 18:18:37 +00:00
Josh Soref cb1303e47a spelling: conjunction 2018-03-11 17:48:37 +00:00
Alex Dadgar 4173834231 Enable more linters 2017-09-26 15:26:33 -07:00
Michael Schurter 67d154a274 Test driver network advertisement and checks 2017-06-21 17:19:08 -07:00
Michael Schurter b9bfb84b53 Implement DriverNetwork and Service.AddressMode
Ideally DriverNetwork would be fully populated in Driver.Prestart, but
Docker doesn't assign the container's IP until you start the container.

However, it's important to setup the port env vars before calling
Driver.Start, so Prestart should populate that.
2017-06-21 17:19:08 -07:00
Michael Schurter fd9bef768f Move task env into execcontext
Also inject PATH into rkt commands since we're no longer appending host
env vars for it.
2017-05-23 13:53:34 -07:00
Michael Schurter 37b148fb60 Add PortMap to struct returned by Driver.Prestart
Moves env.Builder out of drivers entirely so one less thing to worry
about when implementing driver plugins.
2017-05-23 13:53:34 -07:00
Michael Schurter d2c08ff24b Refactor TaskEnvironment into Builder and TaskEnv 2017-05-23 13:53:33 -07:00
Michael Schurter 10b6610e56 Functional consul template env file support 2017-05-23 13:45:14 -07:00
Alex Dadgar ba70cc4f01 Merge branch 'master' into f-bolt-db 2017-05-09 11:11:55 -07:00
Kate Taggart 2fb6301b37 responding to feedback on PR: remove Region from Node struct, some grammatical niceties. 2017-05-03 12:45:59 -07:00
Kate Taggart af22cb722e I think I did it. 2017-05-03 12:45:59 -07:00
Kate Taggart 9fa97c38a0 just found out dc and region are going to be a bit different, as Region didn't previously exist in the Node struct. 2017-05-03 12:45:59 -07:00
Alex Dadgar b94f855326 boltDB database for client state 2017-05-01 14:50:34 -07:00
Michael Schurter caf317e3f2 Use a DriverAbility to expose Exec functionality 2017-04-19 12:42:47 -07:00
Michael Schurter 745ad9521f Move ScriptExecutor to driver 2017-04-19 12:42:47 -07:00
Alex Dadgar 4ecebe7d8c Proper reference counting through task restarts
This PR fixes an issue in which the reference count on a Docker image
would become inflated through task restarts.
2017-03-25 17:05:53 -07:00
Michael Schurter aef3c2e380 Handle createdResourcs=nil
Combined with b522c472fdf this fixes #2256

Without these two commits in place upgrades to 0.5.3 panics.
2017-01-31 10:51:32 -08:00
Alex Dadgar 44d63318a4 Add nil guard 2017-01-31 09:19:59 -08:00
Michael Schurter a28fc4798a Merge pull request #2215 from hashicorp/b-2211-host-env
Fix image based drivers having host env vars set
2017-01-20 13:16:44 -08:00
Michael Schurter 295c2bf74a Merge pull request #2186 from hashicorp/f-driver-cleanup
Add Cleanup method to Driver interface
2017-01-20 13:02:14 -08:00
Michael Schurter f94210b4bc Fix image based drivers having host env vars set
Add detailed tests for GetTaskEnv to avoid this issue happening again!

Fixes #2211
2017-01-18 10:27:03 -08:00
Michael Schurter 578272b7f2 Add CreatedResources.Remove and use it 2017-01-17 16:41:59 -08:00
Michael Schurter a3a3656dbb Switch to use recoverable errors from Cleanup
TaskRunner handles retrying but Cleanup handles all of CreatedResources.
2017-01-13 16:46:08 -08:00
Michael Schurter dc68aa1a5a Return errors from cleanup and let TaskRunner retry 2017-01-12 17:21:54 -08:00
Diptanu Choudhury 6809a4b104 Added executorconfig 2017-01-12 15:47:58 -08:00
Diptanu Choudhury b1d0078db5 Filter executor log messages 2017-01-12 11:54:19 -08:00
Michael Schurter 4d081490e6 Add Cleanup method to Driver interface
Cleanup can be used for cleaning up resources created by drivers to run
a task. Initially the Docker driver is the only user (to remove
downloaded images).
2017-01-11 17:23:33 -08:00
Alex Dadgar 2be221d664 Don't retrieve Driver Stats if unsupported
This PR makes us only try to collect stats once if the Driver doesn't
support collecting stats.

Fixes https://github.com/hashicorp/nomad/issues/1986
2017-01-09 13:47:06 -08:00
Michael Schurter 65fb580216 Fix inconsistent task env setting
Consolidate task environment building in GetTaskEnv since it can
determine what kind of filesystem isolation is used.

This means drivers no longer have to manipulate task environment paths.
2017-01-06 12:19:32 -08:00
Michael Schurter 3ea09ba16a Move chroot building into TaskRunner
* Refactor AllocDir to have a TaskDir struct per task.
* Drivers expose filesystem isolation preference
* Fix lxc mounting of `secrets/`
2017-01-05 16:31:49 -08:00
Alex Dadgar 8d5f0fea69 Merge pull request #2128 from hashicorp/f-dispatch
Nomad Constructor Jobs and Dispatch
2017-01-06 05:22:49 +08:00
Michael Schurter ea92cd102a Append host env vars on every task env 2016-12-20 12:24:24 -08:00
Alex Dadgar b1883daae8 Use new combined meta data function in env 2016-12-16 10:45:09 -08:00
Michael Schurter 770ed703d0 Add Driver.Prestart method
The Driver.Prestart method currently does very little but lays the
foundation for where lifecycle plugins can interleave execution _after_
task environment setup but _before_ the task starts.

Currently Prestart does two things:

* Any driver specific task environment building
* Download Docker images

This change also attaches a TaskEvent emitter to Drivers, so they can
emit events during task initialization.
2016-12-02 11:03:48 -08:00
Alex Dadgar 747c3e7e68 Agent 2016-11-01 10:39:15 -07:00
Alex Dadgar 41b5679015 Advertise signalling abilities 2016-10-19 15:06:23 -07:00
Alex Dadgar 53eeec9bc1 Merge pull request #1801 from hashicorp/f-signals
Consul-template signal change mode
2016-10-18 11:23:47 -07:00
Michael Schurter ca5439eca1 Add NOMAD_JOB_NAME to environment 2016-10-11 11:20:42 -07:00
Alex Dadgar 280af8f4d1 Docker + Qemu 2016-10-10 11:47:04 -07:00
Alex Dadgar 12de69a66f Struct and parse 2016-09-21 11:31:09 -07:00
Alex Dadgar 6702a29071 Vault token threaded 2016-09-14 13:30:01 -07:00
Alex Dadgar eef786dd9d Secret dir materialized in alloc/task directory 2016-09-02 12:44:05 -07:00
Alex Dadgar 2c8dd8bbd3 Revert "Introduce a Secret/ directory" 2016-09-01 17:23:15 -07:00
Alex Dadgar 1ed454dd60 Merge pull request #1671 from hashicorp/f-secret-dir2
Introduce a Secret/ directory
2016-09-01 09:56:17 -07:00
Alex Dadgar 0626eb9619 environment variables 2016-08-31 13:56:11 -07:00