Commit graph

14760 commits

Author SHA1 Message Date
Mahmood Ali ce97689b79 drivers/mock: implement nomad exec interface 2019-05-09 16:49:08 -04:00
Mahmood Ali 13de875750 implemment streaming exec handling in driver grpc handlers
Also add a helper that converts the adapts the high level interface to the
low-level interface of nomad exec interfaces.
2019-05-09 16:49:08 -04:00
Mahmood Ali f47d3d5f8a add nomad streaming exec core data structures and interfaces
In this commit, we add two driver interfaces for supporting `nomad exec`
invocation:

* A high level `ExecTaskStreamingDriver`, that operates on io reader/writers.
  Drivers should prefer using this interface
* A low level `ExecTaskStreamingRawDriver` that operates on the raw stream of
  input structs; useful when a driver delegates handling to driver backend (e.g.
  across RPC/grpc).

The interfaces are optional for a driver, as `nomad exec` support is opt-in.
Existing drivers continue to compile without exec support, until their
maintainer add such support.

Furthermore, we create protobuf structures to represent exec stream entities:
`ExecTaskStreamingRequest` and `ExecTaskStreamingResponse`.  We aim to reuse the
protobuf generated code as much as possible, without translation to avoid
conversion overhead.

`ExecTaskStream` abstract fetching and sending stream entities.  It's influenced
by the grpc bi-directional stream interface, to avoid needing any adapter.  I
considered using channels, but the asynchronisity and concurrency makes buffer
reuse too complicated, which would put more pressure on GC and slows exec operation.
2019-04-30 14:02:29 -04:00
Mahmood Ali 40e62a6f17 Add ACL capabilities for nomad exec
This adds `alloc-exec` capability to allow operator to execute command into a
running task.  Furthermore, it adds `alloc-node-exec` capability, required when
the alloc task is raw_exec or a driver with no FSIsolation.
2019-04-30 14:02:16 -04:00
Mahmood Ali 7131ac3eef drivers/mock: extract command related operations
Extract command parsing and execution mocking into a separate struct.  Also,
allow mocking of different fs_isolation for testing.
2019-04-30 14:02:16 -04:00
Mahmood Ali fc72fff0ed test helper for registering jobs with acl
Test helper that allows registration of jobs when ACL is activated.
2019-04-30 10:23:56 -04:00
Mahmood Ali 446f06721d aux: helper method that returns token as well as ACL policy
This helper returns the token as well as the ACL policy, to be used in a later
commit for logging the token info associated with nomad exec invocation.
2019-04-30 10:23:56 -04:00
Charlie Voiselle 4c2dd47ddb
Merge pull request #5630 from jweissig/patch-5
Fixed typo
2019-04-30 08:59:16 -04:00
Mahmood Ali b18c748c41 update 0.9.1 changelog 2019-04-30 07:48:00 -04:00
Danielle 03762458ed
Merge pull request #5614 from hashicorp/dani/we-released-a-hotfix
Update CHANGELOG and version.go to reflect 0.9.1/0.9.2 split
2019-04-30 10:35:22 +02:00
Danielle Lancashire ee97edc065 Update master to reflect 0.9.1/0.9.2 split 2019-04-30 10:33:41 +02:00
Justin Weissig 4e017f5ed1
Fixed typo
Fixed typo: "dots wil be"/"dots will be".
2019-04-30 00:04:17 -07:00
Yishan Lin d00da5b827
Merge pull request #5618 from hashicorp/yishan/revised-readme
yishan/revised readme
2019-04-29 15:46:20 -07:00
Yishan Lin f96866f7ee Addressed Dani's comments. 2019-04-29 10:22:27 -07:00
Preetha Appan 78c3e55198
Update 0.9.1 links in website 2019-04-29 11:46:50 -05:00
Danielle 6249f2864f
Merge pull request #5623 from jweissig/patch-4
Fixed Typo
2019-04-29 10:25:45 +02:00
Justin Weissig 1c71ffa9dc
Fixed Typo
Fixed typo: scheduilng/scheduling.
2019-04-29 00:41:53 -07:00
Mahmood Ali 23474c5568
Merge pull request #5622 from hashicorp/b-divest-api-20190427
divest /api from rest of nomad internal packages
2019-04-28 16:24:08 -04:00
Mahmood Ali f920efb962 divest /api from nomad/structs
The API package needs to be independent from rest of nomad packages, to
avoid leaking internal packages and dependencies (e.g. raft, ugorji,
etc)
2019-04-28 13:32:26 -04:00
Mahmood Ali 64a45e0e1e check that /api doesn't import internal nomad pkgs 2019-04-28 13:32:26 -04:00
Lang Martin da91b14c90 CHANGELOG.md add fix line for #2619 2019-04-26 16:33:17 -04:00
Yishan Lin 3476b45f42 Update README.md 2019-04-26 11:48:45 -07:00
Yishan Lin a9fb32fa35 Update README.md 2019-04-26 11:48:13 -07:00
Yishan Lin d8d18b1b29 Update README.md 2019-04-26 11:46:30 -07:00
Yishan Lin 752b78558e Reordered guide lists + formatting with users/content. 2019-04-26 11:45:43 -07:00
Yishan Lin 102f3e84cd updated hyperlinks. 2019-04-26 11:43:16 -07:00
Yishan Lin 77a7565218 Update README.md 2019-04-26 11:28:09 -07:00
Lang Martin 371014b781
Merge pull request #5553 from hashicorp/b-fingerprinter-manual-config
client fingerprinter doesn't overwrite manual configuration
2019-04-26 12:55:34 -04:00
Mahmood Ali b3fac6ef74
Merge pull request #5370 from hashicorp/f-windows-volume-binds
drivers/docker: Support volumes field in Windows
2019-04-26 10:12:42 -04:00
Danielle 6ff474cf90
Merge pull request #5612 from hashicorp/dani/alloc-signal-docs
Docs for `nomad alloc signal`
2019-04-26 14:21:23 +02:00
Danielle 79515496cb
Merge pull request #5515 from hashicorp/dani/f-alloc-signal
allocs: Add nomad alloc signal command
2019-04-26 14:21:05 +02:00
Danielle Lancashire a8880f9643 alloc_signal: Add autcompletion and cmd tests 2019-04-26 12:47:53 +02:00
Mahmood Ali 530d03c510
Merge pull request #5616 from hashicorp/b-retry-logmon-start-errs-2
logmon: recover from shutting down call locally
2019-04-25 21:42:09 -04:00
Mahmood Ali bf0a09e270 retry grpc unavailable errors even if not shutting down 2019-04-25 18:39:17 -04:00
Mahmood Ali 81841e8528 try checking process status 2019-04-25 18:16:13 -04:00
Mahmood Ali fc78521f29 add logging about attempts 2019-04-25 18:09:36 -04:00
Mahmood Ali e6ca8641a8 try sleeping for stop signal to take effect 2019-04-25 17:16:29 -04:00
Mahmood Ali ff3a095015 add a test that simulates logmon dying during Start() call 2019-04-25 16:41:17 -04:00
Mahmood Ali bbac73883c logmon: retry starting logmon if it exits
Retry if we detect shutting down during Start() api call is started,
locally.
2019-04-25 15:10:16 -04:00
Mahmood Ali b51f00a7f3 logmon client to handle grpc closing errors 2019-04-25 14:32:24 -04:00
Preetha 4461b4c73e
Update CHANGELOG.md 2019-04-25 12:51:07 -05:00
Mahmood Ali 0ee771b020 driver/docker: Support volumes field in Windows
Support Docker `volumes` field in Windows.  Previously, volumes parser
assumed some Unix-ism (e.g. didn't expect `:` in mount paths).
Here, we use the Docker parser to identify host and container paths.

Docker parsers use different validation logic from our previous unix
implementation: Docker parser accepts single path as a volume entry
(parsing it as a container path with auto-created volume) and enforces
additional checks (e.g. validity of mode).  Thereforce, I opted to use
Docker parser only for Windows, and keep Nomad's linux parser to
preserve current behavior.
2019-04-25 09:02:44 -04:00
Mahmood Ali d314db54d8 vendor docker/docker volume utils 2019-04-25 08:55:21 -04:00
Danielle Lancashire d46d3ebeb4 docs: Fix copy/paste error in alloc-restart docs 2019-04-25 14:40:09 +02:00
Danielle Lancashire 78350bfd77 docs: Add nomad alloc signal docs 2019-04-25 14:39:49 +02:00
Mahmood Ali 81d30a8c11
Merge pull request #5609 from hashicorp/b-docker-tty-logs
driver/docker: collect tty container logs
2019-04-25 08:28:32 -04:00
Danielle Lancashire 3409e0be89 allocs: Add nomad alloc signal command
This command will be used to send a signal to either a single task within an
allocation, or all of the tasks if <task-name> is omitted. If the sent signal
terminates the allocation, it will be treated as if the allocation has crashed,
rather than as if it was operator-terminated.

Signal validation is currently handled by the driver itself and nomad
does not attempt to restrict or validate them.
2019-04-25 12:43:32 +02:00
Danielle 1cfb03ac4f
Merge pull request #5597 from hashicorp/dani/doc-alloc-stop
Documentation for `nomad alloc stop`
2019-04-25 12:23:09 +02:00
Mahmood Ali df2b579c6b driver/docker: collect tty container logs
Fixes https://github.com/hashicorp/nomad/issues/5475

When container is a tty container, we need to get raw terminal output
without any additional processing.
2019-04-24 22:01:51 -04:00
Arshneet Singh 78d49eda7c
Merge pull request #5602 from hashicorp/normalized-plan
Normalize plan to increase the plan apply throughput
2019-04-24 14:07:26 -07:00