Go to file
Seth Hoenig 51a2212d3d
client: sandbox go-getter subprocess with landlock (#15328)
* client: sandbox go-getter subprocess with landlock

This PR re-implements the getter package for artifact downloads as a subprocess.

Key changes include

On all platforms, run getter as a child process of the Nomad agent.
On Linux platforms running as root, run the child process as the nobody user.
On supporting Linux kernels, uses landlock for filesystem isolation (via go-landlock).
On all platforms, restrict environment variables of the child process to a static set.
notably TMP/TEMP now points within the allocation's task directory
kernel.landlock attribute is fingerprinted (version number or unavailable)
These changes make Nomad client more resilient against a faulty go-getter implementation that may panic, and more secure against bad actors attempting to use artifact downloads as a privilege escalation vector.

Adds new e2e/artifact suite for ensuring artifact downloading works.

TODO: Windows git test (need to modify the image, etc... followup PR)

* landlock: fixup items from cr

* cr: fixup tests and go.mod file
2022-12-07 16:02:25 -06:00
.changelog client: sandbox go-getter subprocess with landlock (#15328) 2022-12-07 16:02:25 -06:00
.circleci build: update to go1.19.3 (#15099) 2022-11-01 15:54:49 -05:00
.github pin build/release pipeline to ubuntu 20.04 (#15348) 2022-11-21 14:08:45 -05:00
.release Prepare for next release 2022-11-22 12:56:29 -05:00
.semgrep Pre forwarding authentication (#15417) 2022-12-06 14:44:03 -05:00
.tours Make number of scheduler workers reloadable (#11593) 2022-01-06 11:56:13 -05:00
acl rename SecureVariables to Variables throughout 2022-08-26 16:06:24 -04:00
api acl: added type to ACL Auth Method stub (#15480) 2022-12-06 14:47:05 +01:00
ci ci: use groups of tests in gha (#15018) 2022-10-27 09:02:58 -05:00
client client: sandbox go-getter subprocess with landlock (#15328) 2022-12-07 16:02:25 -06:00
command acl: added type to ACL Auth Method stub (#15480) 2022-12-06 14:47:05 +01:00
contributing make: add target cl for create changelog entry (#15186) 2022-11-08 09:43:32 -06:00
demo demo/docs: update demo of Kadalu CSI Plugin (#13610) 2022-07-06 10:24:34 -04:00
dev docs: swap master for main in Nomad repo 2021-03-08 14:26:31 -05:00
drivers exec: allow running commands from host volume (#14851) 2022-11-11 09:51:15 -05:00
e2e client: sandbox go-getter subprocess with landlock (#15328) 2022-12-07 16:02:25 -06:00
helper client: sandbox go-getter subprocess with landlock (#15328) 2022-12-07 16:02:25 -06:00
integrations spelling: registrations 2018-03-11 18:40:53 +00:00
internal/testing/apitests cleanup: replace TypeToPtr helper methods with pointer.Of (#14151) 2022-08-17 18:26:34 +02:00
jobspec api: remove `mapstructure` tags from`Port` struct (#12916) 2022-11-08 11:26:28 +01:00
jobspec2 template: error on missing key (#15141) 2022-11-04 13:23:01 -04:00
lib Add command "nomad tls" (#14296) 2022-11-22 14:12:07 -05:00
nomad bugfix: make sure streaming endpoints are only registered once (#15484) 2022-12-07 17:01:45 +01:00
plugins Add mount propagation to protobuf definition of mounts (#15096) 2022-11-17 18:14:59 -05:00
scheduler scheduler: create placements for non-register MRD (#15325) 2022-11-25 12:45:34 -05:00
scripts build: update to go1.19.3 (#15099) 2022-11-01 15:54:49 -05:00
terraform terraform: update installed versions of HashiCorp tools. (#13635) 2022-07-07 16:12:19 +02:00
testutil Add command "nomad tls" (#14296) 2022-11-22 14:12:07 -05:00
tools make: add target cl for create changelog entry (#15186) 2022-11-08 09:43:32 -06:00
ui [ui] Policies UI (#13976) 2022-12-06 12:45:36 -05:00
version Prepare for next release 2022-11-22 12:56:29 -05:00
website client: sandbox go-getter subprocess with landlock (#15328) 2022-12-07 16:02:25 -06:00
.git-blame-ignore-revs ignore b0a20b4dc965a38b0c843f47c16685ccad7439da (#13648) 2022-07-07 15:16:18 -07:00
.gitattributes Remove invalid gitattributes 2018-02-14 14:47:43 -08:00
.gitignore ci: use groups of tests in gha (#15018) 2022-10-27 09:02:58 -05:00
.go-version build: update to go1.19.3 (#15099) 2022-11-01 15:54:49 -05:00
.golangci.yml build: update linters (#15063) 2022-10-27 15:02:30 -05:00
.semgrepignore build: disable semgrep on structs.go for now 2022-02-01 10:09:49 -06:00
CHANGELOG.md changelogs for 1.3.8 and 1.2.15 2022-11-22 12:57:55 -05:00
CODEOWNERS ensure engineering has merge authority on build pipeline (#15350) 2022-11-21 14:30:02 -05:00
GNUmakefile Prepare for next release 2022-11-22 12:56:29 -05:00
LICENSE [COMPLIANCE] Update MPL 2.0 LICENSE (#14884) 2022-10-13 08:43:12 -04:00
README.md readme: remove Gitter lobby link. (#14195) 2022-08-22 10:33:20 +02:00
Vagrantfile tools: update virtualbox networking configuration (#11561) 2021-11-24 10:45:58 -05:00
build_linux_arm.go gofmt all the files 2021-10-01 10:14:28 -04:00
go.mod client: sandbox go-getter subprocess with landlock (#15328) 2022-12-07 16:02:25 -06:00
go.sum client: sandbox go-getter subprocess with landlock (#15328) 2022-12-07 16:02:25 -06:00
main.go client: sandbox go-getter subprocess with landlock (#15328) 2022-12-07 16:02:25 -06:00
main_test.go Adding initial skeleton 2015-06-01 13:46:21 +02:00

README.md

Nomad License: MPL 2.0 Discuss

HashiCorp Nomad logo

Nomad is a simple and flexible workload orchestrator to deploy and manage containers (docker, podman), non-containerized applications (executable, Java), and virtual machines (qemu) across on-prem and clouds at scale.

Nomad is supported on Linux, Windows, and macOS. A commercial version of Nomad, Nomad Enterprise, is also available.

Nomad provides several key features:

  • Deploy Containers and Legacy Applications: Nomads flexibility as an orchestrator enables an organization to run containers, legacy, and batch applications together on the same infrastructure. Nomad brings core orchestration benefits to legacy applications without needing to containerize via pluggable task drivers.

  • Simple & Reliable: Nomad runs as a single binary and is entirely self contained - combining resource management and scheduling into a single system. Nomad does not require any external services for storage or coordination. Nomad automatically handles application, node, and driver failures. Nomad is distributed and resilient, using leader election and state replication to provide high availability in the event of failures.

  • Device Plugins & GPU Support: Nomad offers built-in support for GPU workloads such as machine learning (ML) and artificial intelligence (AI). Nomad uses device plugins to automatically detect and utilize resources from hardware devices such as GPU, FPGAs, and TPUs.

  • Federation for Multi-Region, Multi-Cloud: Nomad was designed to support infrastructure at a global scale. Nomad supports federation out-of-the-box and can deploy applications across multiple regions and clouds.

  • Proven Scalability: Nomad is optimistically concurrent, which increases throughput and reduces latency for workloads. Nomad has been proven to scale to clusters of 10K+ nodes in real-world production environments.

  • HashiCorp Ecosystem: Nomad integrates seamlessly with Terraform, Consul, Vault for provisioning, service discovery, and secrets management.

Quick Start

Testing

See Learn: Getting Started for instructions on setting up a local Nomad cluster for non-production use.

Optionally, find Terraform manifests for bringing up a development Nomad cluster on a public cloud in the terraform directory.

Production

See Learn: Nomad Reference Architecture for recommended practices and a reference architecture for production deployments.

Documentation

Full, comprehensive documentation is available on the Nomad website: https://www.nomadproject.io/docs

Guides are available on HashiCorp Learn.

Contributing

See the contributing directory for more developer documentation.