Commit Graph

13853 Commits

Author SHA1 Message Date
Nick Ethier d8a0265e68
client: batch initial fingerprinting in plugin manangers
drivermanager: fix pr comments/feedback
2018-12-18 22:56:19 -05:00
Nick Ethier 7d23cbf448
client/drivermananger: fixup issues from rebase and address PR comments 2018-12-18 22:55:38 -05:00
Nick Ethier 1543335710
tr: deregister task handler on cleanup 2018-12-18 22:55:38 -05:00
Nick Ethier 82175d1328
client/drivermananger: add driver manager
The driver manager is modeled after the device manager and is started by the client.
It's responsible for handling driver lifecycle and reattachment state, as well as
processing the incomming fingerprint and task events from each driver. The mananger
exposes a method for registering event handlers for task events that is used by the
task runner to update the server when a task has been updated with an event.

Since driver fingerprinting has been implemented by the driver manager, it is no
longer needed in the fingerprint mananger and has been removed.
2018-12-18 22:55:18 -05:00
Alex Dadgar bf464f7a6a
Merge pull request #5015 from hashicorp/f-plugin-versions
Add plugin API versioning to plugin loader and plugins
2018-12-18 16:49:02 -08:00
Alex Dadgar bc55ec81b5 fix docker launching plugins 2018-12-18 16:48:01 -08:00
Alex Dadgar 730a6f5b9a lint 2018-12-18 16:48:00 -08:00
Alex Dadgar 4c57d2ec4d Add plugin API versioning to plugin loader and plugins 2018-12-18 16:48:00 -08:00
Alex Dadgar 1dabde6e0b base fixes 2018-12-18 16:48:00 -08:00
Alex Dadgar 74e7e0fba7 protos 2018-12-18 16:48:00 -08:00
Alex Dadgar 9d1403d617
Merge pull request #5002 from hashicorp/b-task-config-resources
Convert driver resource to AllocatedTaskResource
2018-12-18 16:46:34 -08:00
Preetha 2a8896892e
Merge pull request #5024 from hashicorp/f-affinities-e2e
Affinities e2e tests
2018-12-18 18:41:16 -06:00
Preetha Appan 29d2f8172a
Affinities e2e tests 2018-12-18 18:39:45 -06:00
Danielle Tomlinson 0edc65631a
Merge pull request #5007 from hashicorp/dani/f-allocrunner-async
allocrunner: Async api for shutdown/destroy/update
2018-12-19 01:26:41 +01:00
Alex Dadgar 8efac7ec81 Fix unit tests + upgrade pathing resources 2018-12-18 15:50:44 -08:00
Alex Dadgar b8268d9a46 Lint 2018-12-18 15:50:44 -08:00
Alex Dadgar 66cf3156b2 LinuxResources doesn't use task.Resources 2018-12-18 15:50:44 -08:00
Alex Dadgar 327b551b39 Drivers 2018-12-18 15:50:11 -08:00
Alex Dadgar b653ae2af7 utilities 2018-12-18 15:48:52 -08:00
Alex Dadgar b9ee03b2c1 protos 2018-12-18 15:48:52 -08:00
Nathan R Valentine 7fdf78c57c change retry_join to a map for easier tag handling 2018-12-18 15:48:17 -08:00
Danielle Tomlinson 792ddc5db1
Merge pull request #5021 from hashicorp/dani/rand-tasks
taskrunner: Use a random suffix for Task Config
2018-12-19 00:39:55 +01:00
Danielle Tomlinson 95a0c4fb29 taskrunner: Use a random suffix for Task Config
The RestartCount is not really suitable for use as a source of
uniqueness within task invocations as it is not monotonic, and interacts
with the restart stanza in a users config, so conflates restarts due to
task failures, with restarts due to enviromental changes, such as consul
template or vault secrets changing.

Here we instead use a substring from a uuid, which is more random than
we strictly need, but is nicer than rolling our own random string
generator here.
2018-12-19 00:38:54 +01:00
Danielle Tomlinson 1be0170ebe client: Update tests for async destroy 2018-12-18 23:38:34 +01:00
Danielle Tomlinson d6eb084d8a allocrunner: Drop and log updates after closing waitCh 2018-12-18 23:38:34 +01:00
Danielle Tomlinson 0d91285cd6 allocrunner: Documentation for ShutdownCh/DestroyCh 2018-12-18 23:38:34 +01:00
Danielle Tomlinson f2bb13818e fixup: Log when we detect out of order updates 2018-12-18 23:38:33 +01:00
Danielle Tomlinson 986fde0f5a allocrunner: Handle updates asynchronously
This creates a new buffered channel and goroutine on the allocrunner for
serializing updates to allocations. This allows us to take updates off
the routine that is used from processing updates from the server,
without having complicated machinery for tracking update lifetimes, or
other external synchronization.

This results in a nice performance improvement and signficantly better
throughput on batch changes such as preempting a large number of jobs
for a larger placement.
2018-12-18 23:38:33 +01:00
Danielle Tomlinson f3fa9d1406 gc: Wait for allocrunners to be destroyed 2018-12-18 23:38:33 +01:00
Danielle Tomlinson cb78a90f40 client: Async API for shutdown/destroy allocrunners 2018-12-18 23:38:33 +01:00
Danielle Tomlinson d1fbac1aad allocrunner: Async shutdown and destroy
This commit reduces the locking required to shutdown or destroy
allocrunners, and allows parallel shutdown and destroy of allocrunners during
shutdown.
2018-12-18 23:38:33 +01:00
Omar Khawaja b1ab008abf
Commenting out dead link to demo.nomadproject.io (#5017)
* removing dead link

* comment out header and footer to UI demo

* adding reference to link back and commenting it out
2018-12-18 17:26:51 -05:00
Preetha 6c51232f55
Merge pull request #5012 from hashicorp/f-e2e-provisioning
Terraform configs for e2e tests
2018-12-18 13:45:58 -06:00
Preetha Appan fb8980b46d
added readme 2018-12-18 13:37:03 -06:00
Michael Lange fa4333056c
Merge pull request #4981 from hashicorp/b-ui-hide-stats-graphs-for-non-running-resources
UI: Hide stats graphs for non running resources
2018-12-18 11:15:39 -08:00
Danielle Tomlinson 0cb8450659
Merge pull request #5016 from hashicorp/dani/b-docker-delete-task-on-destroy
docker: Delete Task on Destroy
2018-12-18 18:22:36 +01:00
Danielle Tomlinson b61da13c20 docker: Delete Task on Destroy
Currently the docker driver does not remove tasks from its state map
when destroying the task, which leads to issues when restarting tasks in
place, and leaks expired handles over time.
2018-12-18 15:53:31 +01:00
Michael Lange e1e5aa21ff
Merge pull request #4994 from hashicorp/b-ui-dots-in-tasks
UI: Bugs around dots in task/task-group/driver names
2018-12-17 15:50:31 -08:00
Preetha Appan 1ba4674ce2
suggestions from code review 2018-12-17 15:06:22 -06:00
Jack Pearkes 844d981e77
Terraform configs for e2e tests 2018-12-17 11:40:09 -06:00
Danielle Tomlinson d9174d8dcf
Merge pull request #4989 from hashicorp/dani/b-client-update-race-condition
client: Give a copy of clientconfig to allocrunner
2018-12-17 10:49:46 +01:00
Danielle Tomlinson 53aa1bc198
Merge pull request #5004 from hashicorp/dani/f-hook-errors
client: Emit TaskEvents when task hooks fail
2018-12-17 10:42:57 +01:00
Danielle Tomlinson a50ea29da4 taskrunner: Use hook errors for artifacts 2018-12-17 10:39:38 +01:00
Mahmood Ali 2d2c562e18 Remove implicit check
I intended to remove this line in 29ef7ecf2372f980d12a9900e1b2a351568dd415 - see my notes there for details.
2018-12-16 09:14:26 -05:00
Mahmood Ali 5e2d5ef4c6 ci: use gotestsum for running tests
`gotestsum` has user friendlier output that emits final summary, also it can emit junit xml file for
automated analysis instead of current format that should significantly
ease automated analysis of CI.
2018-12-15 21:41:11 -05:00
Mahmood Ali 56dfdd0874
tests: fix rkt command environment (#5011)
The environment variables needed for envoking `rkt` command line
should include host PATH (to access `iptables`).

Given that the command runs outside the VM, untrusted task environment
variables should NOT be honored here.

We do this already with `rkt`, but the change is quite subtle to miss
when refactoring.
2018-12-15 20:25:36 -05:00
Mahmood Ali 168749ffd1
Merge pull request #5008 from hashicorp/b-docker-test-20181214
Fix flakiness in docker tests
2018-12-15 16:03:00 -05:00
Mahmood Ali d58e38e912 tests: avoid implicitly asserting clean shutdown
The assertion here is causing many spurious failures that aren't
actually relevant to the test itself.

We are tracking the cause for this failure independently, and it would
make more sense to have a dedicated test for clean shutdown.
2018-12-15 15:30:09 -05:00
Mahmood Ali e4f44b9be5 testes: remove TestDockerDriver_Kill
We already have two other Kill tests (e.g.
TestDockerDriver_Start_Kill_Wait and
TestDockerDriver_Start_KillTimeout), so don't need yet another flaky
test.
2018-12-15 15:03:56 -05:00
Mahmood Ali 990a7d6776 driver/docker: stopping a dead container not error 2018-12-15 15:03:56 -05:00