Commit Graph

26 Commits

Author SHA1 Message Date
Seth Hoenig 8b05efcf88 consul/connect: Add support for Connect terminating gateways
This PR implements Nomad built-in support for running Consul Connect
terminating gateways. Such a gateway can be used by services running
inside the service mesh to access "legacy" services running outside
the service mesh while still making use of Consul's service identity
based networking and ACL policies.

https://www.consul.io/docs/connect/gateways/terminating-gateway

These gateways are declared as part of a task group level service
definition within the connect stanza.

service {
  connect {
    gateway {
      proxy {
        // envoy proxy configuration
      }
      terminating {
        // terminating-gateway configuration entry
      }
    }
  }
}

Currently Envoy is the only supported gateway implementation in
Consul. The gateay task can be customized by configuring the
connect.sidecar_task block.

When the gateway.terminating field is set, Nomad will write/update
the Configuration Entry into Consul on job submission. Because CEs
are global in scope and there may be more than one Nomad cluster
communicating with Consul, there is an assumption that any terminating
gateway defined in Nomad for a particular service will be the same
among Nomad clusters.

Gateways require Consul 1.8.0+, checked by a node constraint.

Closes #9445
2021-01-25 10:36:04 -06:00
Tim Gross 724f82e32d docs: add metrics and API docs to RPC contributors checklist 2021-01-04 15:41:12 -05:00
Michael Schurter 772ab5502d build: upgrade from Go 1.15.5 -> 1.15.6
Also updated scripts/update_golang_version.sh to handle new CircleCI
file layout.
2021-01-04 08:34:00 -08:00
Kris Hicks 0a05a5e3b4
README: Align with Consul README (#9681)
* Add alt text to Nomad logo
* Hoist Resources
* Use description from nomadproject.io
* Getting Started -> Quick Start
* Add production cluster section to Quick Start
* Remove "Who Uses Nomad"
* Move Contributing to contributing/README.md
* Add link to homepage from hero image
2020-12-18 09:38:34 -08:00
davemay99 52c8a0f37d Update golang version table 2020-12-08 08:51:34 -05:00
Seth Hoenig 8fe8f7ba1f docs: note manual jobspec parsing generally no longer required 2020-11-30 09:46:40 -06:00
Tim Gross d26f30bba8
docs: add contributor docs for issue labels (#8723) 2020-08-24 10:19:57 -04:00
Lang Martin 717741a4cb
doc: nomad debug cli (#8278)
* doc: nomad debug cli

* CHANGELOG

* website/data/docs-navigation: add debug to navigation

* contributing/checklist-command: add website nav link to checklist
2020-06-25 13:48:27 -04:00
Seth Hoenig d5ad580d5c structs: fix compatibility between api and nomad/structs proxy definitions
The field names within the structs representing the Connect proxy
definition were not the same (nomad/structs/ vs api/), causing the
values to be lost in translation for the 'nomad job inspect' command.

Since the field names already shipped in v0.11.0 we cannot simply
fix the names. Instead, use the json struct tag on the structs/ structs
to remap the name to match the publicly expose api/ package on json
encoding.

This means existing jobs from v0.11.0 will continue to work, and
the JSON API for job submission will remain backwards compatible.
2020-04-13 15:59:45 -06:00
Seth Hoenig 8209eb4d80 doc: add a reminder about field order for diff comparisons
After spending hours re-learning this lesson more than once,
update the jobspec contribution guide with a reminder about
how to configure FieldDiff comparisons.
2020-03-25 09:33:27 -06:00
Lang Martin a0a6766740 CSI: Scheduler knows about CSI constraints and availability (#6995)
* structs: piggyback csi volumes on host volumes for job specs

* state_store: CSIVolumeByID always includes plugins, matches usecase

* scheduler/feasible: csi volume checker

* scheduler/stack: add csi volumes

* contributing: update rpc checklist

* scheduler: add volumes to State interface

* scheduler/feasible: introduce new checker collection tgAvailable

* scheduler/stack: taskGroupCSIVolumes checker is transient

* state_store CSIVolumeDenormalizePlugins comment clarity

* structs: remote TODO comment in TaskGroup Validate

* scheduler/feasible: CSIVolumeChecker hasPlugins improve comment

* scheduler/feasible_test: set t.Parallel

* Update nomad/state/state_store.go

Co-Authored-By: Danielle <dani@hashicorp.com>

* Update scheduler/feasible.go

Co-Authored-By: Danielle <dani@hashicorp.com>

* structs: lift ControllerRequired to each volume

* state_store: store plug.ControllerRequired, use it for volume health

* feasible: csi match fast path remove stale host volume copied logic

* scheduler/feasible: improve comments

Co-authored-by: Danielle <dani@builds.terrible.systems>
2020-03-23 13:58:29 -04:00
Lang Martin 88316208a0 csi: server-side plugin state tracking and api (#6966)
* structs: CSIPlugin indexes jobs acting as plugins and node updates

* schema: csi_plugins table for CSIPlugin

* nomad: csi_endpoint use vol.Denormalize, plugin requests

* nomad: csi_volume_endpoint: rename to csi_endpoint

* agent: add CSI plugin endpoints

* state_store_test: use generated ids to avoid t.Parallel conflicts

* contributing: add note about registering new RPC structs

* command: agent http register plugin lists

* api: CSI plugin queries, ControllerHealthy -> ControllersHealthy

* state_store: copy on write for volumes and plugins

* structs: copy on write for volumes and plugins

* state_store: CSIVolumeByID returns an unhealthy volume, denormalize

* nomad: csi_endpoint use CSIVolumeDenormalizePlugins

* structs: remove struct errors for missing objects

* nomad: csi_endpoint return nil for missing objects, not errors

* api: return meta from Register to avoid EOF error

* state_store: CSIVolumeDenormalize keep allocs in their own maps

* state_store: CSIVolumeDeregister error on missing volume

* state_store: CSIVolumeRegister set indexes

* nomad: csi_endpoint use CSIVolumeDenormalizePlugins tests
2020-03-23 13:58:29 -04:00
Lang Martin af857ae660 docs contributing: checklist-command 2020-03-23 13:58:29 -04:00
Lang Martin 2eb288426c docs: update checklist-rpc-endpoint 2020-03-23 13:58:29 -04:00
Drew Bailey 680f5301b8
add note to check for job diff 2020-03-04 12:06:32 -05:00
Michael Schurter f1d493f46a docs: updating from go1.13 -> 1.14 for 0.11 2020-03-02 12:09:13 -08:00
Lang Martin d75f77b8b5 Update checklist-rpc-endpoint.md
Add an ACL bullet point
2020-02-14 13:10:28 -05:00
Michael Schurter a04facf0ef docs: explicitly callout checking for prior art
Also normalize formatting.
2020-02-12 08:07:07 -08:00
Michael Schurter b5869240c6 docs: add Go versioning policy 2020-01-27 08:18:58 -08:00
Seth Hoenig fd75d8a487
Merge pull request #6447 from hashicorp/contributing-doc-tweaks
contributing: fix typo & append RPC checklist
2019-10-08 14:54:56 -05:00
Seth Hoenig e2b34550d6 contributing: fix typo & append RPC checklist 2019-10-08 19:28:35 +00:00
Michael Schurter 3d3a445ee7 docs: add sidebar to jobspec checklist 2019-10-08 10:55:53 -07:00
Lang Martin b9f90701ea checklist NodeDeregisterBatchRequestType must go at the end 2019-07-10 13:56:20 -04:00
Lang Martin 909f3b0534 new file: contributing/checklist-rpc-endpoint.md 2019-07-10 13:56:20 -04:00
Michael Schurter b2d62096d0 docs: checklists don't support numbered lists
The order didn't really make sense anyway.
2019-06-12 15:35:48 -07:00
Michael Schurter 6c116add0f docs: move dev docs to contributing 2019-06-07 15:41:13 -07:00