luxolus/open-nomad
2
0
Fork 0
Code Issues 1 Pull requests Packages Releases Wiki Activity
13449 commits 1 branch 0 tags 311 MiB
Commit graph

13449 commits

This branch
This branch
All branches
Author SHA1 Message Date
Nick Ethier 117b9e6584
drivers: support recoverable errors in the plugin RPC layer 2018-11-19 22:59:15 -05:00
Nick Ethier 8f8698b3e1
docker: started work on porting docker driver to new plugin framework 2018-11-19 22:59:15 -05:00
Nick Ethier 9e64ce7d73
docker: properly launch docker logger process 2018-11-19 22:59:12 -05:00
Sevag Hanssian 6ab55108b8
webiste -> website 2018-11-19 18:50:42 -08:00
Mahmood Ali d744e71fa9 add a missing no errorassertion 2018-11-19 21:44:00 -05:00
Mahmood Ali b93643cd96 Fix a panic related to batch GC 
`deleteJobVersions` does concurrent modifications to iterated items
while iterating, by deleting job versions while it's iterating on them,
 2018-11-19 20:59:45 -05:00
Mahmood Ali bff9c3b3e9 Reproduce a panic related to batch GC 
Test case that reproduces a panic with the following stacktrace:

```
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
	panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x38 pc=0x1149715]

goroutine 35 [running]:
testing.tRunner.func1(0xc0001e2200)
	/usr/local/Cellar/go/1.11.2/libexec/src/testing/testing.go:792 +0x387
panic(0x167e400, 0x1c43a30)
	/usr/local/Cellar/go/1.11.2/libexec/src/runtime/panic.go:513 +0x1b9
github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-immutable-radix.(*Iterator).Next(0xc0003a4080, 0x17f7ba0, 0x0, 0xc0002e74a0, 0xc0003a0510, 0xc0003a0530, 0xc0003a0530)
	/go/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-immutable-radix/iter.go:81 +0xa5
github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.(*radixIterator).Next(0xc0003a0420, 0x1756059, 0xb)
	/go/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/txn.go:634 +0x2e
github.com/hashicorp/nomad/nomad/state.(*StateStore).deleteJobVersions(0xc00028f7d0, 0x2711, 0xc0002e7680, 0xc000392100, 0xc0003a4040, 0x0)
	/go/src/github.com/hashicorp/nomad/nomad/state/state_store.go:1130 +0x1a1
github.com/hashicorp/nomad/nomad/state.(*StateStore).DeleteJobTxn(0xc00028f7d0, 0x2711, 0x175334f, 0x7, 0xc000306810, 0x2f, 0xc000392100, 0x0, 0x0)
	/go/src/github.com/hashicorp/nomad/nomad/state/state_store.go:1102 +0x46c
github.com/hashicorp/nomad/nomad/state.TestStateStore_DeleteJobTxn_BatchDeletes.func1(0xc000392100, 0x1777ce0, 0xc000392100)
	/go/src/github.com/hashicorp/nomad/nomad/state/state_store_test.go:1705 +0x1a2
github.com/hashicorp/nomad/nomad/state.(*StateStore).WithWriteTransaction(0xc00028f7d0, 0xc0000d5e48, 0x0, 0x0)
	/go/src/github.com/hashicorp/nomad/nomad/state/state_store.go:3953 +0x79
github.com/hashicorp/nomad/nomad/state.TestStateStore_DeleteJobTxn_BatchDeletes(0xc0001e2200)
	/go/src/github.com/hashicorp/nomad/nomad/state/state_store_test.go:1703 +0x685
testing.tRunner(0xc0001e2200, 0x1777138)
	/usr/local/Cellar/go/1.11.2/libexec/src/testing/testing.go:827 +0xbf
created by testing.(*T).Run
	/usr/local/Cellar/go/1.11.2/libexec/src/testing/testing.go:878 +0x353
```
 2018-11-19 20:58:32 -05:00
Michael Schurter 56ed4f01be vault: fix panic by checking for nil secret 
Vault's RenewSelf(...) API may return (nil, nil). We failed to check if
secret was nil before attempting to use it.

RenewSelf:
e3eee5b4fb/api/auth_token.go (L138-L155)

Calls ParseSecret:
e3eee5b4fb/api/secret.go (L309-L311)

If anyone has an idea on how to test this I didn't see any options. We
use a real Vault service, so there's no opportunity to mock the
response.
 2018-11-19 17:07:59 -08:00
Michael Schurter 88577fe083 client.rpc: don't log errors on shutdown 2018-11-19 16:39:30 -08:00
Michael Schurter 5bd744ac3d client: support graceful shutdowns 
Client.Shutdown now blocks until all AllocRunners and TaskRunners have
exited their Run loops. Tasks are left running.
 2018-11-19 16:39:30 -08:00
Michael Lange fd35d37f43
Merge pull request #4860 from hashicorp/b-ui-allocation-detail-empty-state 
UI: Allocation detail empty state
 2018-11-19 12:43:39 -08:00
Michael Lange 796967c58d Add an empty state to the allocation detail task list 2018-11-19 12:18:14 -08:00
Michael Lange b252ede5b6
Merge pull request #4850 from hashicorp/f-ui-links-on-error-pages 
UI: Escape hatch links on error pages
 2018-11-19 12:17:36 -08:00
Omar Khawaja 181381f456
change instrumented webserver job file (#4897) 2018-11-17 01:10:44 -05:00
Omar Khawaja ba2514149f
clarification regarding Nomad client generating token vs retrieving (#4890) 2018-11-16 19:52:31 -05:00
Mahmood Ali 9479015f51
Merge pull request #4884 from hashicorp/f-alloc-devices-cli 
Report alloc device statistics in API and CLI
 2018-11-16 18:04:54 -05:00
Mahmood Ali 6f9126f475 show Device Stats header in alloc status 2018-11-16 17:34:37 -05:00
Mahmood Ali 00ffd02ced Show stable order of device attributes 2018-11-16 17:34:37 -05:00
Mahmood Ali f139234372 address review comments 2018-11-16 17:13:01 -05:00
Michael Schurter dac17f5337
Merge pull request #4843 from hashicorp/b-driver-env 
client: interpolate driver configurations
 2018-11-16 12:45:43 -06:00
Preetha 225f1531cb
Update CHANGELOG.md 2018-11-16 12:31:54 -06:00
Preetha 5f094633fa
Merge pull request #4889 from hashicorp/f-service-meta 
Pass service metadata "external-source" for consul UI integration
 2018-11-16 12:24:21 -06:00
Preetha Appan 18708d3f0b
Pass service metadata "external-source" for consul UI integration 2018-11-16 11:28:56 -06:00
Mahmood Ali 33c96a803a tweak whitespace in device stats output 2018-11-16 10:37:39 -05:00
Mahmood Ali 159b8f866a Display device stats in nomad alloc status 2018-11-16 10:26:32 -05:00
Mahmood Ali d88a3f8413 Prepare to reuse device resources printing 2018-11-16 10:26:32 -05:00
Mahmood Ali f72e599ee7 Populate alloc stats API with device stats 
This change makes few compromises:

* Looks up the devices associated with tasks at look up time.  Given
that `nomad alloc status` is called rarely generally (compared to stats
telemetry and general job reporting), it seems fine.  However, the
lookup overhead grows bounded by number of `tasks x total-host-devices`,
which can be significant.

* `client.Client` performs the task devices->statistics lookup.  It
passes self to alloc/task runners so they can look up the device statistics
allocated to them.
  * Currently alloc/task runners are responsible for constructing the
entire RPC response for stats
  * The alternatives for making task runners device statistics aware
don't seem appealing (e.g. having task runners contain reference to hostStats)

* On the alloc aggregation resource usage, I did a naive merging of task device statistics.
  * Personally, I question the value of such aggregation, compared to
costs of struct duplication and bloating the response - but opted to be
consistent in the API.
  * With naive concatination, device instances from a single device group used by separate tasks in the alloc, would be aggregated in two separate device group statistics.
 2018-11-16 10:26:32 -05:00
Preetha 7873b212af
Merge pull request #4882 from sportebois/f-update-docs-env-stanza-coercion 
Change misleading boolean example in env stanza coercion section (#4820)
 2018-11-16 08:00:56 -06:00
Michael Schurter 0cdb188ae4 tests: fix tests post-rebase 2018-11-15 17:40:56 -08:00
Michael Schurter 59f106ecee client/tr: add a bit of context to envbuilder errors 2018-11-15 16:26:25 -08:00
Michael Schurter 742f8775ba client: remove old proxy references from comments 2018-11-15 16:26:25 -08:00
Michael Schurter 2d0b44c3b4 client: test more env key variations 2018-11-15 16:26:25 -08:00
Michael Schurter 8bcd90d78d client: add new nested variables to task's hcl ctx 
The error messages are really bad, but it's extremely difficult to
produce good error messages without the original HCL.
 2018-11-15 16:26:25 -08:00
Michael Schurter 3b8fe566a9 docs: document new interpolation behavior and options 2018-11-15 16:26:25 -08:00
Michael Schurter 5e51e2c2d5 client: turn env into nested objects for task configs 2018-11-15 16:25:57 -08:00
Michael Schurter f8cdd561f0 client: interpolate driver configurations 
Also add missing SetDriverNetwork calls.
 2018-11-15 16:25:57 -08:00
Michael Schurter 4d92603340 boltdd: return error on use-after-Close 
Return the same error as boltdb instead of panic'ing.
 2018-11-15 14:15:37 -08:00
Michael Schurter 6f3712ed48 gofmt -s -w command/helper_stats_test.go 
Fixes the static checks build
 2018-11-15 14:14:05 -08:00
Preetha Appan cba7ed58e0
update consul template 2018-11-15 14:15:12 -06:00
Danielle Tomlinson 8bf17fe22d
Merge pull request #4875 from hashicorp/f-constraints 
scheduler: Make != constraints more flexible
 2018-11-15 11:04:21 -08:00
Mahmood Ali 1ec6f551a6
Merge pull request #4879 from hashicorp/f-node-devices-cli 
Show device stats in nomad CLI
 2018-11-15 14:02:22 -05:00
Danielle Tomlinson 890bde3482 Add changelog entry for != operator 2018-11-15 11:00:32 -08:00
Danielle Tomlinson 023ef40cd0 docs: Add is_set/is_not_set 2018-11-15 11:00:32 -08:00
Danielle Tomlinson 9c72dafc95 scheduler: Add is_set/is_not_set constraints 
This adds constraints for asserting that a given attribute or value
exists, or does not exist. This acts as a companion to =, or !=
operators, e.g:

```hcl
constraint {
        attribute = "${attrs.type}"
        operator  = "!="
        value     = "database"
}

constraint {
        attribute = "${attrs.type}"
        operator  = "is_set"
}
```
 2018-11-15 11:00:32 -08:00
Sébastien Portebois bf494cb86c Change misleading boolean example in env stanza coercion section 2018-11-15 13:12:58 -05:00
Preetha Appan 4f7a78ba5c
review comments 2018-11-15 11:32:22 -06:00
Mahmood Ali 24b37e0aaf Display StatsObject nested objects as well 2018-11-15 08:09:54 -05:00
Mahmood Ali ee9353fbd6 Use disk display format for devices 2018-11-14 22:13:23 -05:00
Mahmood Ali 0712be643f Print verbose device in nomad node status -stats 2018-11-14 22:13:23 -05:00
Mahmood Ali 93e8fc53f9 device stats summary in node status 
Sample output with a mock device:

```
Host Resource Utilization
CPU             Memory          Disk
2651/26400 MHz  9.6 GiB/16 GiB  98 GiB/234 GiB

Device Resource Utilization
nomad/file/mock[README.md]    511 bytes
nomad/file/mock[e2e.go]       239 bytes
nomad/file/mock[e2e_test.go]  128 bytes

Allocations
No allocations placed
```
 2018-11-14 22:13:23 -05:00