Commit Graph

17524 Commits

Author SHA1 Message Date
Chris Baker ffd79583f6
Merge pull request #7474 from hashicorp/f-scaling-changes-from-review
more testing for scaling API
2020-03-24 15:32:10 -05:00
Chris Baker c638c2c352 update RPC scaling endpoint tests to use renamed 'scale' policy disposition 2020-03-24 20:18:12 +00:00
Chris Baker 5979d6a81e more testing for ScalingPolicy, mainly around parsing and canonicalization for Min/Max 2020-03-24 19:43:50 +00:00
Chris Baker ea050b20c7 test coverage for scaling policy min/max parsing 2020-03-24 19:28:24 +00:00
Lang Martin 8bd0405f33
csi: return an empty result list from plugins & volumes without `type`, not an error (#7471) 2020-03-24 14:28:28 -04:00
Drew Bailey 2430954a9d
Merge pull request #7429 from hashicorp/docs-audit-logging
docs: audit logging
2020-03-24 14:24:31 -04:00
Drew Bailey 4b74084aba
enterprise index page update 2020-03-24 14:22:37 -04:00
Tim Gross b0dbef3221
CSI: HTTP API docs (#7435)
HTTP API documentation for the `/plugin` and `/volume` endpoints, to
support CSI.
2020-03-24 14:21:45 -04:00
Drew Bailey 2f5e77dba4
Audit logging documentation
docs for audit logging

enterprise only note

enterprise page audit blurb

add audit to sidebar

add general options audit info

fix name for docs-navigation

link to docs

more docs

add audit log entry examples

fix audit link

clean up audit log docs
2020-03-24 14:20:27 -04:00
Chris Baker aa5beafe64 Job.Scale should not result in job update or eval create if args.Count == nil
plus tests
2020-03-24 17:36:06 +00:00
Tim Gross 913da68296
csi: remove client from plugin on client node update (#7462)
Plugins track the client nodes where they are placed. On client
updates, remove the client from the plugin tracking if the client is
no longer running an instance of that controller/node plugin.

Extends the state store tests to ensure deregistration works as
expected and that controllers and nodes are being tracked
independently.
2020-03-24 13:26:31 -04:00
Chris Baker 9e530e167d
Merge pull request #7409 from hashicorp/scaling-api
Scaling API changes
2020-03-24 11:02:09 -05:00
Tim Gross 4ef4a604c2
Documentation for CSI command line and jobspec (#7439)
Includes:

* changes `nomad alloc status` and `nomad node status`
* changes to `volume` block in jobspec
* new `csi_plugin` block in jobspec
* new `nomad volume register/deregister/status` commands
* new `nomad plugin status` command
2020-03-24 11:09:46 -04:00
Chris Baker bc13bfb433 bad conversion between api.ScalingPolicy and structs.ScalingPolicy meant
that we were throwing away .Min if provided
2020-03-24 14:39:06 +00:00
Chris Baker ab4d174319 added new int64ToPtr method to api/util to avoid pulling in other packages 2020-03-24 14:39:05 +00:00
Chris Baker 606c79b320 add acl validation to Scaling.ListPolicies and Scaling.GetPolicy 2020-03-24 14:39:05 +00:00
Chris Baker f6ec5f9624 made count optional during job scaling actions
added ACL protection in Job.Scale
in Job.Scale, only perform a Job.Register if the Count was non-nil
2020-03-24 14:39:05 +00:00
Chris Baker 41b002eecc wip: ACL checking for RPC Job.ScaleStatus 2020-03-24 14:39:05 +00:00
Chris Baker ca49563c94 added new ACL capabilities related to autoscaling:
- read-job-scaling
- scale-job
- list-scaling-policies
- read-scaling-policy

updated the read and right policy dispositions, added the new autoscaler disposition
2020-03-24 14:38:58 +00:00
Lang Martin bd22afd003
csi: volume deregister fails for volumes actively in use (#7445)
* nomad/structs/csi: add InUse to CSIVolume

* nomad/state/state_store: block volume deregistration for in use vols
2020-03-24 10:10:44 -04:00
Tim Gross 599e6f4605
Merge pull request #7450 from hashicorp/b-csi-plugin-health-counts
csi: fix missing health count from volume list stub
2020-03-24 10:07:07 -04:00
Chris Baker 233db5258a changes to Canonicalize, Validate, and api->struct conversion so that tg.Count, tg.Scaling.Min/Max are well-defined with reasonable defaults.
- tg.Count defaults to tg.Scaling.Min if present (falls back on previous default of 1 if Scaling is absent)
- Validate() enforces tg.Scaling.Min <= tg.Count <= tg.Scaling.Max

modification in ApiScalingPolicyToStructs, api.TaskGroup.Validate so that defaults are handled for TaskGroup.Count and
2020-03-24 13:57:17 +00:00
Chris Baker 5373d503c7 scaling api: put api.* objects in agreement with structs.* objects 2020-03-24 13:57:16 +00:00
Chris Baker 00092a6c29 fixed http endpoints for job.register and job.scalestatus 2020-03-24 13:57:16 +00:00
Chris Baker f9876a487e finished Job.ScaleStatus RPC, need to work on http endpoint 2020-03-24 13:57:16 +00:00
Chris Baker 925b59e1d2 wip: scaling status return, almost done 2020-03-24 13:57:15 +00:00
James Rasell f125b5fb2d scaling: ensure min and max int64s are in toplevel of block. 2020-03-24 13:57:15 +00:00
Chris Baker 42270d862c wip: some tests still failing
updating job scaling endpoints to match RFC, cleaning up the API object as well
2020-03-24 13:57:14 +00:00
Chris Baker abc7a52f56 finished refactoring state store, schema, etc 2020-03-24 13:57:14 +00:00
Chris Baker 116aa98ed7 wip: removed some commented junk from scaling poc 2020-03-24 13:57:13 +00:00
Luiz Aoqui 47d35489d6 wip: use testify in job scaling tests 2020-03-24 13:57:13 +00:00
Luiz Aoqui d4b6e4b258 wip: add tests for job scale method 2020-03-24 13:57:12 +00:00
Luiz Aoqui c74c01a643 wip: add scaling policies methods to the client 2020-03-24 13:57:12 +00:00
Chris Baker 3d54f1feba wip: added Enabled to ScalingPolicyListStub, removed JobID from body of scaling request 2020-03-24 13:57:12 +00:00
Chris Baker 84953a1ed7 wip: remove PolicyOverride from scaling request 2020-03-24 13:57:11 +00:00
Chris Baker 024d203267 wip: added tests for client methods around group scaling 2020-03-24 13:57:11 +00:00
Chris Baker 1c5c2eb71b wip: add GET endpoint for job group scaling target 2020-03-24 13:57:10 +00:00
Luiz Aoqui 6699d5e536 wip: add job scale endpoint in client 2020-03-24 13:57:10 +00:00
Chris Baker 179ab68258 wip: added job.scale rpc endpoint, needs explicit test (tested via http now) 2020-03-24 13:57:09 +00:00
Chris Baker 8453e667c2 wip: working on job group scaling endpoint 2020-03-24 13:55:20 +00:00
Chris Baker 6665d0bfb0 wip: added policy get endpoint, added UUID to policy 2020-03-24 13:55:20 +00:00
Chris Baker 8626e32557 wip: test for scaling policy parsing 2020-03-24 13:55:19 +00:00
Chris Baker 1f844a54f9 wip: was incorrectly parsing ScalingPolicy 2020-03-24 13:55:19 +00:00
Chris Baker 9c2560ceeb wip: upsert/delete scaling policies on job upsert/delete 2020-03-24 13:55:18 +00:00
Chris Baker 65d92f1fbf WIP: adding ScalingPolicy to api/structs and state store 2020-03-24 13:55:18 +00:00
Mahmood Ali 6ddf3d1742
Merge pull request #7414 from hashicorp/b-network-mode-change
Detect network mode change
2020-03-24 09:46:40 -04:00
Tim Gross fa01a6ea59 csi: fix missing health count from volume list stub 2020-03-24 09:42:59 -04:00
Lang Martin 0847cb513c
csi: volume/plugin list should return an empty array, not nil (#7443)
* nomad/csi_endpoint: return an empty list, not nil

* nomad/csi_endpoint_test: volume list returns non-nil
2020-03-23 21:21:40 -04:00
Lang Martin d994990ef0
csi: the scheduler allows a job with a volume write claim to be updated (#7438)
* nomad/structs/csi: split CanWrite into health, in use

* scheduler/scheduler: expose AllocByID in the state interface

* nomad/state/state_store_test

* scheduler/stack: SetJobID on the matcher

* scheduler/feasible: when a volume writer is in use, check if it's us

* scheduler/feasible: remove SetJob

* nomad/state/state_store: denormalize allocs before Claim

* nomad/structs/csi: return errors on claim, with context

* nomad/csi_endpoint_test: new alloc doesn't look like an update

* nomad/state/state_store_test: change test reference to CanWrite
2020-03-23 21:21:04 -04:00
Drew Bailey 2a0f3badfa
Merge pull request #7444 from hashicorp/rename-auditor
make auditor interface more explicit
2020-03-23 20:23:28 -04:00