Commit graph

15640 commits

Author SHA1 Message Date
Danielle Tomlinson 886486e694
ci: Migrate to CircleCI
This commit provides an initial migration of general testing CI
infrastructure to CircleCI.

It uses CircleCI 2.1 paramereterised jobs to provide two base
configurations: a vm based `test-machine`, and docker based
`test-container`.

Jobs that require root, docker, or other similar features require the
machine based jobs, but others should be ran using the `test-container` package
as they are both cheaper and faster to run.
2019-08-12 18:41:36 +02:00
Mahmood Ali 20c44b4214
Merge pull request #6068 from hashicorp/r-always-honor-gotags
make: always honor GO_TAGS in dev
2019-08-12 11:22:48 -04:00
Tim Gross 58c395aa9c
tests: partial revert of splitting-out command pkg tests (#6101)
GOTEST_PKG_EXCLUDE overrides GOTEST_PKG entirely, so having both in
the same test run isn't supported and results in a whole lot of extra
tests being run.
2019-08-12 11:03:04 -04:00
Danielle 1fd9ef61f0
Merge pull request #6100 from hashicorp/f-host-volumes
Host Volumes Support: Rollup Edition
2019-08-12 16:58:37 +02:00
Danielle Lancashire 7e6c8e5ac1
Copy documentation to api/tasks 2019-08-12 16:22:27 +02:00
Danielle Lancashire dec1a58b47
fixup rebase 2019-08-12 15:41:14 +02:00
Danielle Lancashire b38c1d810e
job_endpoint: Validate volume permissions 2019-08-12 15:39:09 +02:00
Danielle Lancashire 5f734652f2
acl: Add HostVolume ACLs
This adds an initial implementation of ACLs for HostVolumes.

Because HostVolumes are a cluster-wide resource, they cannot be tied to
a namespace, thus here we allow similar wildcard definitions based on
their names, tied to a set of capabilities.

Initially, the only available capabilities are deny, or mount. These
may be extended in the future to allow read-fs, mount-readonly and
similar capabilities.
2019-08-12 15:39:09 +02:00
Danielle Lancashire 7208a7ab88
command: Cleanup node-status 2019-08-12 15:39:09 +02:00
Danielle Lancashire 333fdd723b
cli: Display host volume info in nomad node status 2019-08-12 15:39:09 +02:00
Danielle Lancashire 6caac09743
api: Add HostVolumeInfo to response parsing 2019-08-12 15:39:09 +02:00
Danielle Lancashire 33db40d4e6
structs: Document VolumeMount 2019-08-12 15:39:08 +02:00
Danielle Lancashire 861caa9564
HostVolumeConfig: Source -> Path 2019-08-12 15:39:08 +02:00
Danielle Lancashire e132a30899
structs: Unify Volume and VolumeRequest 2019-08-12 15:39:08 +02:00
Danielle fc53283489
Update scheduler/feasible.go
Co-Authored-By: Mahmood Ali <mahmood@hashicorp.com>
2019-08-12 15:39:08 +02:00
Danielle Lancashire 6ef8d5233e
client: Add volume_hook for mounting volumes 2019-08-12 15:39:08 +02:00
Danielle Lancashire 073836ec67
scheduler: Add a feasability checker for Host Vols 2019-08-12 15:39:08 +02:00
Danielle Lancashire 01f3fe13fb
api: Allow submission of jobs with volumes 2019-08-12 15:39:08 +02:00
Danielle Lancashire 063e4240c1
client: Add parsing and registration of HostVolume configuration 2019-08-12 15:39:08 +02:00
Danielle Lancashire 6d7b417e54
structs: Add declarations of basic structs for volume support 2019-08-12 15:39:08 +02:00
Danielle Lancashire b45bd36230
jobspec: Add Volume and VolumeMount declarations 2019-08-12 15:39:07 +02:00
Nick Ethier 1871c1edbc
Add sidecar_task stanza parsing (#6104)
* jobspec: breakup parse.go into smaller files

* add sidecar_task parsing to jobspec and api

* jobspec: combine service parsing logic for task and group service stanzas

* api: use slice of ConsulUpstream values instead of pointers
2019-08-09 15:18:53 -04:00
Tim Gross 8e9d01ade8
tests: break out command, command/agent on Travis (#6094)
The command and command/agent packages are taking 5+ minutes on Travis
and this contributes to build timeouts. While this doesn't address
underlying issues, breaking these out can reduce re-runs until that
work is done.
2019-08-09 08:25:11 -04:00
Preetha 1d543290af
Merge pull request #6090 from hashicorp/f-task-kind
Add field "kind" to task for use in connect tasks
2019-08-08 14:40:12 -05:00
Nick Ethier d5b07b4ac7
Revert CNI auto-download (#6095)
Revert CNI auto-download
2019-08-08 15:26:49 -04:00
Nick Ethier 7806f4c597
Revert "client: add autofetch for CNI plugins"
This reverts commit 0bd157cc3b04fb090dd0d54affcae71496102ce8.
2019-08-08 15:10:19 -04:00
Nick Ethier 7d28ece8de
Revert "client: remove debugging lines"
This reverts commit 54ce4d1f7ef4913cb12c03dbc98bcd903f7787c9.
2019-08-08 14:52:52 -04:00
defau1t aae9c4b368 Fix Rkt driver requirements list (#6093) 2019-08-08 12:03:41 -05:00
Buck Doyle 4394e90fec Update changelog with #6087 2019-08-08 09:29:10 -05:00
Buck Doyle 58f6e5d286
Fix navigation via clicking recent allocation row (#6087)
This fixes the recent allocation rows to navigate when clicking within them
rather than just on the link, which matches the cursor behaviour.
2019-08-08 09:26:46 -05:00
Preetha Appan a393ea79e8
Add field "kind" to task for use in connect tasks 2019-08-07 18:43:36 -05:00
Jasmine Dahilig bf6f97edfb remove #6064 from 0.10.0 changelog 2019-08-07 10:55:26 -07:00
Jasmine Dahilig 8d980edd2e
add create and modify timestamps to evaluations (#5881) 2019-08-07 09:50:35 -07:00
Michael Schurter 5d3b0faf75 changelog: add #6003 2019-08-07 08:04:16 -07:00
Michael Schurter 3e4796799a
Merge pull request #6003 from pete-woods/add-job-status-metrics
nomad: add job status metrics
2019-08-07 08:02:16 -07:00
Michael Schurter d2862b33e6
Merge pull request #6045 from hashicorp/f-connect-groupservice
consul: add Connect structs
2019-08-06 15:43:38 -07:00
Michael Schurter ef9d100d2f
Merge pull request #6082 from hashicorp/b-vault-deadlock
vault: fix deadlock in SetConfig
2019-08-06 15:30:17 -07:00
Michael Schurter ecb1a65bb9
Merge pull request #6077 from hashicorp/b-vault-revlock
vault: fix race in accessor revocations
2019-08-06 14:28:47 -07:00
Michael Schurter b8e127b3c0 vault: ensure SetConfig calls are serialized
This is a defensive measure as SetConfig should only be called serially.
2019-08-06 11:17:10 -07:00
Michael Schurter 5022341b27 vault: fix deadlock in SetConfig
This seems to be the minimum viable patch for fixing a deadlock between
establishConnection and SetConfig.

SetConfig calls tomb.Kill+tomb.Wait while holding v.lock.
establishConnection needs to acquire v.lock to exit but SetConfig is
holding v.lock until tomb.Wait exits. tomb.Wait can't exit until
establishConnect does!

```
  SetConfig -> tomb.Wait
     ^              |
     |              v
  v.lock <- establishConnection
```
2019-08-06 10:40:14 -07:00
Michael Schurter 17fd82d6ad consul: add Connect structs
Refactor all Consul structs into {api,structs}/services.go because
api/tasks.go didn't make sense anymore and structs/structs.go is
gigantic.
2019-08-06 08:15:07 -07:00
Jasmine Dahilig cf8dc65382 update changelog 2019-08-05 17:40:39 -07:00
Michael Schurter d0a83eb818 vault: fix race in accessor revocations 2019-08-05 15:08:04 -07:00
Preetha 31c5630e8b
Merge pull request #6076 from hashicorp/f-comment-stable-field
Add more comments to clarify job.Stable field
2019-08-05 16:49:40 -05:00
Jasmine Dahilig ac488bc9dc
job region defaults to client node region if 'global' or none provided (#6064) 2019-08-05 14:28:02 -07:00
Mahmood Ali b17bac5101 Render consul templates using task env only (#6055)
When rendering a task consul template, ensure that only task environment
variables are used.

Currently, `consul-template` always falls back to host process
environment variables when key isn't a task env var[1].  Thus, we add
an empty entry for each host process env-var not found in task env-vars.

[1] bfa5d0e133/template/funcs.go (L61-L75)
2019-08-05 16:30:47 -04:00
Preetha Appan 8b298621ef
Add more comments to clarify job.Stable field 2019-08-05 15:00:53 -05:00
Mahmood Ali 02a316592c make: always honor GO_TAGS in dev
Our build scripts pass `$(GO_TAGS)` to `-tags` go compile flags, except
for `make dev`, where `$(NOMAD_UI_TAG)` is used.  This change ensures
`make dev` is inline with the rest of makefile targets.

I use the flag primarily to enable the nomad ui using the committed
compiled assets without regenerating them, as I find using stale ui
satisfactory most of the time.
2019-08-02 23:55:52 +08:00
Mahmood Ali f66169cd6a
Merge pull request #6065 from hashicorp/b-nil-driver-exec
Check if driver handle is nil before execing
2019-08-02 09:48:28 -05:00
Mahmood Ali a4670db9b7 Check if driver handle is nil before execing
Defend against tr.getDriverHandle being nil.  Exec handler checks if
task is running, but it may be stopped between check and driver handler
fetching.
2019-08-02 10:07:41 +08:00