open-nomad/nomad
Michael Schurter 0a496c845e
Task API via Unix Domain Socket (#15864)
This change introduces the Task API: a portable way for tasks to access Nomad's HTTP API. This particular implementation uses a Unix Domain Socket and, unlike the agent's HTTP API, always requires authentication even if ACLs are disabled.

This PR contains the core feature and tests but followup work is required for the following TODO items:

- Docs - might do in a followup since dynamic node metadata / task api / workload id all need to interlink
- Unit tests for auth middleware
- Caching for auth middleware
- Rate limiting on negative lookups for auth middleware

---------

Co-authored-by: Seth Hoenig <shoenig@duck.com>
2023-02-06 11:31:22 -08:00
..
deploymentwatcher renamed stanza to block for consistency with other projects (#15941) 2023-01-30 15:48:43 +01:00
drainer cleanup: replace TypeToPtr helper methods with pointer.Of (#14151) 2022-08-17 18:26:34 +02:00
mock rpc: add OIDC login related endpoints. 2023-01-13 13:14:29 +00:00
state acl: make auth method default across all types (#15869) 2023-01-26 14:17:11 +01:00
stream event stream: ensure token expiry is correctly checked for subs. 2022-10-27 13:08:05 -04:00
structs Task API via Unix Domain Socket (#15864) 2023-02-06 11:31:22 -08:00
volumewatcher volumewatcher: prevent panic on nil volume (#15101) 2022-11-01 16:53:10 -04:00
acl.go WI: allow workloads to use RPCs associated with HTTP API (#15870) 2023-01-25 14:33:06 -05:00
acl_endpoint.go Task API via Unix Domain Socket (#15864) 2023-02-06 11:31:22 -08:00
acl_endpoint_test.go acl: return 400 not 404 code when creating an invalid policy. (#16000) 2023-02-01 17:40:15 +01:00
acl_test.go acl: Fix panic when bogus token is passed (#15863) 2023-01-25 10:03:17 -05:00
alloc_endpoint.go metrics: Add RPC rate metrics to endpoints that validate TLS names (#15900) 2023-01-26 15:04:25 -05:00
alloc_endpoint_test.go WI: allow workloads to use RPCs associated with HTTP API (#15870) 2023-01-25 14:33:06 -05:00
autopilot.go autopilot: include only servers from the same region (#15290) 2022-11-17 12:09:36 -05:00
autopilot_oss.go migrate autopilot implementation to raft-autopilot (#14441) 2022-09-01 14:27:10 -04:00
autopilot_test.go autopilot: include only servers from the same region (#15290) 2022-11-17 12:09:36 -05:00
blocked_evals.go cleanup: remove more copies of min/max from helper 2022-08-24 09:56:15 -05:00
blocked_evals_stats.go metrics: even classless blocked evals get metrics 2022-07-15 14:12:44 -05:00
blocked_evals_stats_test.go metrics: even classless blocked evals get metrics 2022-07-15 14:12:44 -05:00
blocked_evals_system.go
blocked_evals_test.go ci: swap ci parallelization for unconstrained gomaxprocs 2022-03-15 12:58:52 -05:00
client_agent_endpoint.go metrics: measure rate of RPC requests that serve API (#15876) 2023-01-25 16:37:24 -05:00
client_agent_endpoint_test.go Data race fixes in tests and a new semgrep rule (#14594) 2022-09-15 10:35:08 -07:00
client_alloc_endpoint.go metrics: measure rate of RPC requests that serve API (#15876) 2023-01-25 16:37:24 -05:00
client_alloc_endpoint_test.go Data race fixes in tests and a new semgrep rule (#14594) 2022-09-15 10:35:08 -07:00
client_csi_endpoint.go metrics: Add rate metrics to Client CSI endpoints (#15905) 2023-01-26 16:40:58 -05:00
client_csi_endpoint_test.go metrics: Add rate metrics to Client CSI endpoints (#15905) 2023-01-26 16:40:58 -05:00
client_fs_endpoint.go metrics: measure rate of RPC requests that serve API (#15876) 2023-01-25 16:37:24 -05:00
client_fs_endpoint_test.go ci: swap ci parallelization for unconstrained gomaxprocs 2022-03-15 12:58:52 -05:00
client_rpc.go core: remove all traces of unused protocol version 2022-02-18 16:12:36 -08:00
client_rpc_test.go ci: swap ci parallelization for unconstrained gomaxprocs 2022-03-15 12:58:52 -05:00
client_stats_endpoint.go metrics: measure rate of RPC requests that serve API (#15876) 2023-01-25 16:37:24 -05:00
client_stats_endpoint_test.go ci: swap ci parallelization for unconstrained gomaxprocs 2022-03-15 12:58:52 -05:00
config.go GC: ensure no leakage of evaluations for batch jobs. (#15097) 2023-01-31 13:32:14 -05:00
consul.go consul: Removed unused ConsulUsage.Kinds. (#11303) 2022-09-22 10:07:14 -05:00
consul_oss_test.go consul: Removed unused ConsulUsage.Kinds. (#11303) 2022-09-22 10:07:14 -05:00
consul_policy.go consul: check for acceptable service identity on consul tokens (#15928) 2023-01-27 18:15:51 -06:00
consul_policy_oss_test.go consul: check for acceptable service identity on consul tokens (#15928) 2023-01-27 18:15:51 -06:00
consul_policy_test.go build: run gofmt on all go source files 2022-08-16 11:14:11 -05:00
consul_test.go ci: swap ci parallelization for unconstrained gomaxprocs 2022-03-15 12:58:52 -05:00
core_sched.go GC: ensure no leakage of evaluations for batch jobs. (#15097) 2023-01-31 13:32:14 -05:00
core_sched_test.go GC: ensure no leakage of evaluations for batch jobs. (#15097) 2023-01-31 13:32:14 -05:00
csi_endpoint.go metrics: measure rate of RPC requests that serve API (#15876) 2023-01-25 16:37:24 -05:00
csi_endpoint_test.go remove most static RPC handlers (#15451) 2022-12-02 10:12:05 -05:00
deployment_endpoint.go metrics: Add RPC rate metrics to endpoints that validate TLS names (#15900) 2023-01-26 15:04:25 -05:00
deployment_endpoint_test.go cleanup: replace TypeToPtr helper methods with pointer.Of (#14151) 2022-08-17 18:26:34 +02:00
deployment_watcher_shims.go
drainer_int_test.go cleanup: replace TypeToPtr helper methods with pointer.Of (#14151) 2022-08-17 18:26:34 +02:00
drainer_shims.go
encrypter.go keyring: update handle to state inside replication loop (#15227) 2022-11-17 08:40:12 -05:00
encrypter_test.go keyring: update handle to state inside replication loop (#15227) 2022-11-17 08:40:12 -05:00
endpoints_oss.go provide `RPCContext` to all RPC handlers (#15430) 2022-12-01 10:05:15 -05:00
eval_broker.go Rename `nomad.broker.total_blocked` metric (#15835) 2023-01-20 14:23:56 -05:00
eval_broker_test.go Rename `nomad.broker.total_blocked` metric (#15835) 2023-01-20 14:23:56 -05:00
eval_endpoint.go metrics: Add RPC rate metrics to endpoints that validate TLS names (#15900) 2023-01-26 15:04:25 -05:00
eval_endpoint_test.go eval delete: move batching of deletes into RPC handler and state (#15117) 2022-11-14 14:08:13 -05:00
event_endpoint.go metrics: measure rate of RPC requests that serve API (#15876) 2023-01-25 16:37:24 -05:00
event_endpoint_test.go event stream: ensure token expiry is correctly checked for subs. 2022-10-27 13:08:05 -04:00
fsm.go renamed stanza to block for consistency with other projects (#15941) 2023-01-30 15:48:43 +01:00
fsm_oss.go chore: ensure consistent file naming for non-enterprise files. 2022-01-13 11:32:16 +01:00
fsm_registry_oss.go gofmt all the files 2021-10-01 10:14:28 -04:00
fsm_test.go core: enforce strict steps for clients reconnect (#15808) 2023-01-25 15:53:59 -05:00
heartbeat.go remove most static RPC handlers (#15451) 2022-12-02 10:12:05 -05:00
heartbeat_test.go cleanup: replace TypeToPtr helper methods with pointer.Of (#14151) 2022-08-17 18:26:34 +02:00
job_endpoint.go consul: restore consul token when reverting a job (#15996) 2023-02-01 14:02:45 -06:00
job_endpoint_hook_connect.go renamed stanza to block for consistency with other projects (#15941) 2023-01-30 15:48:43 +01:00
job_endpoint_hook_connect_test.go provide `RPCContext` to all RPC handlers (#15430) 2022-12-01 10:05:15 -05:00
job_endpoint_hook_expose_check.go build: run gofmt on all go source files 2022-08-16 11:14:11 -05:00
job_endpoint_hook_expose_check_test.go build: run gofmt on all go source files 2022-08-16 11:14:11 -05:00
job_endpoint_hook_vault.go cleanup more helper updates (#14638) 2022-09-21 14:53:25 -05:00
job_endpoint_hook_vault_oss.go Support Vault entity aliases (#12449) 2022-04-05 14:18:10 -04:00
job_endpoint_hooks.go renamed stanza to block for consistency with other projects (#15941) 2023-01-30 15:48:43 +01:00
job_endpoint_hooks_test.go servicedisco: implicit constraint for nomad v1.4 when using nsd checks (#14868) 2022-10-11 08:21:42 -05:00
job_endpoint_oss.go scheduler: create placements for non-register MRD (#15325) 2022-11-25 12:45:34 -05:00
job_endpoint_oss_test.go cleanup: replace TypeToPtr helper methods with pointer.Of (#14151) 2022-08-17 18:26:34 +02:00
job_endpoint_test.go renamed stanza to block for consistency with other projects (#15941) 2023-01-30 15:48:43 +01:00
job_endpoint_validators.go cleanup: purge github.com/pkg/errors 2022-04-01 19:24:02 -05:00
job_endpoint_validators_test.go allocrunner: refactor task coordinator (#14009) 2022-08-22 18:38:49 -04:00
keyring_endpoint.go metrics: Add RPC rate metrics to endpoints that validate TLS names (#15900) 2023-01-26 15:04:25 -05:00
keyring_endpoint_test.go keyring: use nanos for `CreateTime` in key metadata (#13849) 2022-07-20 14:46:57 -04:00
leader.go core: add ACL binding rule to replication system. (#15555) 2022-12-16 09:08:00 +01:00
leader_oss.go gofmt all the files 2021-10-01 10:14:28 -04:00
leader_test.go cleanup: remove usage of consul/sdk/testutil/retry (#15609) 2023-01-02 08:06:20 -06:00
merge.go
namespace_endpoint.go metrics: measure rate of RPC requests that serve API (#15876) 2023-01-25 16:37:24 -05:00
namespace_endpoint_test.go acl: Fix panic when bogus token is passed (#15863) 2023-01-25 10:03:17 -05:00
node_endpoint.go Allow wildcard datacenters to be specified in job file (#11170) 2023-02-02 09:57:45 -05:00
node_endpoint_test.go core: enforce strict steps for clients reconnect (#15808) 2023-01-25 15:53:59 -05:00
operator_endpoint.go metrics: measure rate of RPC requests that serve API (#15876) 2023-01-25 16:37:24 -05:00
operator_endpoint_test.go WI: allow workloads to use RPCs associated with HTTP API (#15870) 2023-01-25 14:33:06 -05:00
periodic.go make version checks specific to region (1.4.x) (#14912) 2022-10-17 16:23:51 -04:00
periodic_endpoint.go metrics: measure rate of RPC requests that serve API (#15876) 2023-01-25 16:37:24 -05:00
periodic_endpoint_test.go ci: swap ci parallelization for unconstrained gomaxprocs 2022-03-15 12:58:52 -05:00
periodic_test.go ci: swap ci parallelization for unconstrained gomaxprocs 2022-03-15 12:58:52 -05:00
plan_apply.go keyring: safely handle missing keys and restore GC (#15092) 2022-11-01 15:00:50 -04:00
plan_apply_node_tracker.go Track plan rejection history and automatically mark clients as ineligible (#13421) 2022-07-12 18:40:20 -04:00
plan_apply_node_tracker_test.go Track plan rejection history and automatically mark clients as ineligible (#13421) 2022-07-12 18:40:20 -04:00
plan_apply_oss.go chore: ensure consistent file naming for non-enterprise files. 2022-01-13 11:32:16 +01:00
plan_apply_pool.go
plan_apply_pool_test.go ci: swap ci parallelization for unconstrained gomaxprocs 2022-03-15 12:58:52 -05:00
plan_apply_test.go fix panic from keyring raft entries being written during upgrade (#14821) 2022-10-06 12:47:02 -04:00
plan_endpoint.go metrics: Add RPC rate metrics to endpoints that validate TLS names (#15900) 2023-01-26 15:04:25 -05:00
plan_endpoint_test.go fix deadlock in plan_apply (#13407) 2022-06-23 12:06:27 -04:00
plan_normalization_test.go ci: swap ci parallelization for unconstrained gomaxprocs 2022-03-15 12:58:52 -05:00
plan_queue.go Add missing timer reset (#15134) 2022-11-03 18:57:57 -04:00
plan_queue_test.go ci: swap ci parallelization for unconstrained gomaxprocs 2022-03-15 12:58:52 -05:00
raft_rpc.go
regions_endpoint.go metrics: Add metrics to unauthenticated endpoints (#15899) 2023-01-26 15:05:51 -05:00
regions_endpoint_test.go ci: swap ci parallelization for unconstrained gomaxprocs 2022-03-15 12:58:52 -05:00
rpc.go feat: remove dependency to consul/lib 2022-04-09 13:22:44 +02:00
rpc_rate_metrics.go add metric for count of RPC requests (#15515) 2023-01-24 11:54:20 -05:00
rpc_test.go implement pre-forwarding auth on select RPCs (#15513) 2023-01-24 10:52:07 -05:00
scaling_endpoint.go metrics: measure rate of RPC requests that serve API (#15876) 2023-01-25 16:37:24 -05:00
scaling_endpoint_test.go ci: swap ci parallelization for unconstrained gomaxprocs 2022-03-15 12:58:52 -05:00
search_endpoint.go metrics: measure rate of RPC requests that serve API (#15876) 2023-01-25 16:37:24 -05:00
search_endpoint_oss.go rename SecureVariables to Variables throughout 2022-08-26 16:06:24 -04:00
search_endpoint_test.go ci: swap ci parallelization for unconstrained gomaxprocs 2022-03-15 12:58:52 -05:00
serf.go migrate autopilot implementation to raft-autopilot (#14441) 2022-09-01 14:27:10 -04:00
serf_test.go ci: fix TestNomad_BootstrapExpect_NonVoter test (#14407) 2022-08-30 16:32:54 -04:00
server.go metrics: Add rate metrics to Client CSI endpoints (#15905) 2023-01-26 16:40:58 -05:00
server_setup.go core: move LicenseConfig to shared file (#14247) 2022-08-23 13:44:10 -07:00
server_setup_oss.go migrate autopilot implementation to raft-autopilot (#14441) 2022-09-01 14:27:10 -04:00
server_test.go implement pre-forwarding auth on select RPCs (#15513) 2023-01-24 10:52:07 -05:00
service_registration_endpoint.go metrics: Add RPC rate metrics to endpoints that validate TLS names (#15900) 2023-01-26 15:04:25 -05:00
service_registration_endpoint_test.go deps: update set and test (#14680) 2022-09-26 08:28:03 -05:00
stats_fetcher.go metrics: Add metrics to unauthenticated endpoints (#15899) 2023-01-26 15:05:51 -05:00
stats_fetcher_test.go test: fix concurrent map access in `TestStatsFetcher` (#14496) 2022-09-08 10:41:15 -04:00
status_endpoint.go metrics: Add metrics to unauthenticated endpoints (#15899) 2023-01-26 15:05:51 -05:00
status_endpoint_test.go implement pre-forwarding auth on select RPCs (#15513) 2023-01-24 10:52:07 -05:00
system_endpoint.go metrics: measure rate of RPC requests that serve API (#15876) 2023-01-25 16:37:24 -05:00
system_endpoint_test.go ci: swap ci parallelization for unconstrained gomaxprocs 2022-03-15 12:58:52 -05:00
testing.go ci: swap freeport for portal in packages (#15661) 2023-01-03 11:25:20 -06:00
testing_oss.go gofmt all the files 2021-10-01 10:14:28 -04:00
timetable.go
timetable_test.go ci: swap ci parallelization for unconstrained gomaxprocs 2022-03-15 12:58:52 -05:00
util.go make version checks specific to region (1.4.x) (#14912) 2022-10-17 16:23:51 -04:00
util_test.go make version checks specific to region (1.4.x) (#14912) 2022-10-17 16:23:51 -04:00
variables_endpoint.go metrics: measure rate of RPC requests that serve API (#15876) 2023-01-25 16:37:24 -05:00
variables_endpoint_test.go implement pre-forwarding auth on select RPCs (#15513) 2023-01-24 10:52:07 -05:00
vault.go vault: configure user agent on Nomad vault clients (#15745) 2023-01-10 10:39:45 -06:00
vault_test.go cleanup: replace TypeToPtr helper methods with pointer.Of (#14151) 2022-08-17 18:26:34 +02:00
vault_testing.go vault: detect namespace change in config reload (#14298) 2022-08-24 17:03:29 -04:00
worker.go core: backoff considerably when worker is behind raft (#15523) 2023-01-24 08:56:35 -05:00
worker_string_schedulerworkerstatus.go Make number of scheduler workers reloadable (#11593) 2022-01-06 11:56:13 -05:00
worker_string_workerstatus.go Make number of scheduler workers reloadable (#11593) 2022-01-06 11:56:13 -05:00
worker_test.go core: backoff considerably when worker is behind raft (#15523) 2023-01-24 08:56:35 -05:00