History

Tim Gross 2a6e8be6ba internals documentation with diagrams (#14750 ) This changeset adds new architecture internals documents to the contributing guide. These are intentionally here and not on the public-facing website because the material is not required for operators and includes a lot of diagrams that we can cheaply maintain with mermaid syntax but would involve art assets to have up on the main site that would become quickly out of date as code changes happen and be extremely expensive to maintain. However, these should be suitable to use as points of conversation with expert end users. Included: * A description of Evaluation triggers and expected counts, with examples. * A description of Evaluation states and implicit states. This is taken from an internal document in our team wiki. * A description of how writing the State Store works. This is taken from a diagram I put together a few months ago for internal education purposes. * A description of Evaluation lifecycle, from registration to running Allocations. This is mostly lifted from @lgfa29's amazing mega-diagram, but broken into digestible chunks and without multi-region deployments, which I'd like to cover in a future doc. Also includes adding Deployments to our public-facing glossary. Co-authored-by: Luiz Aoqui <luiz@hashicorp.com> Co-authored-by: Michael Schurter <mschurter@hashicorp.com> Co-authored-by: Seth Hoenig <shoenig@duck.com>		2022-10-03 14:06:41 -04:00
..
architecture-eval-lifecycle.md	internals documentation with diagrams (#14750 )	2022-10-03 14:06:41 -04:00
architecture-eval-states.md	internals documentation with diagrams (#14750 )	2022-10-03 14:06:41 -04:00
architecture-eval-triggers.md	internals documentation with diagrams (#14750 )	2022-10-03 14:06:41 -04:00
architecture-state-store.md	internals documentation with diagrams (#14750 )	2022-10-03 14:06:41 -04:00
cgo.md	docs: describe the cgo dependency	2022-03-09 12:46:57 -06:00
CHANGELOG.md
checklist-command.md
checklist-jobspec.md
checklist-rpc-endpoint.md
golang.md
issue-labels.md
README.md	build: update go version to go1.19.1 (#14653 )	2022-09-22 09:40:01 -05:00
testing.md	docs: API package tests need a binary with your changes (#13029 )	2022-05-16 11:12:54 -04:00

README.md

Nomad Codebase Documentation

This directory contains some documentation about the Nomad codebase, aimed at readers who are interested in making code contributions.

If you're looking for information on using Nomad, please instead refer to the Nomad website.

Developing with Vagrant

A development environment is supplied via Vagrant to make getting started easier.

Install Vagrant
Install Virtualbox
Bring up the Vagrant project
```
$ git clone https://github.com/hashicorp/nomad.git
$ cd nomad
$ vagrant up
```
The virtual machine will launch, and a provisioning script will install the needed dependencies within the VM.
SSH into the VM
```
$ vagrant ssh
```

Developing without Vagrant

Install Go 1.19.1+ (Note: gcc-go is not supported)

Clone this repo

$ git clone https://github.com/hashicorp/nomad.git
$ cd nomad

Bootstrap your environment
```
$ make bootstrap
```
(Optionally) Set a higher ulimit, as Nomad creates many file handles during normal operations
```
$ [ "$(ulimit -n)" -lt 1024 ] && ulimit -n 1024
```
Verify you can run tests
```
$ make test
```

Running a development build

Compile a development binary (see the UI README to include the web UI in the binary)
```
$ make dev
# find the built binary at ./bin/nomad
```
Start the agent in dev mode
```
$ sudo bin/nomad agent -dev
```
(Optionally) Run Consul to enable service discovery and health checks
1. Download Consul
2. Start Consul in dev mode
```
$ consul agent -dev
```

Compiling Protobufs

If in the course of your development you change a Protobuf file (those ending in .proto), you'll need to recompile the protos.

Run make boostrap to install the buf command.
Compile Protobufs
```
$ make proto
```

Building the Web UI

See the UI README for instructions.

Create a release binary

To create a release binary:

$ make prerelease
$ make release
$ ls ./pkg

This will generate all the static assets, compile Nomad for multiple platforms and place the resulting binaries into the ./pkg directory.

API Compatibility

Only the api/ and plugins/ packages are intended to be imported by other projects. The root Nomad module does not follow semver and is not intended to be imported directly by other projects.

Architecture

The code for Nomad's major components is organized as:

api/ provides a Go client for Nomad's HTTP API.
client/ contains Nomad's client agent code.
command/ contains Nomad's CLI code.
nomad/ contains Nomad's server agent code.
ui/ contains Nomad's UI code.
website/ contains Nomad's website and documentation.

The high level control flow for many Nomad actions (via the CLI or UI) are:

# Read actions:
Client -> HTTP API -> RPC -> StateStore

# Actions that change state:
Client -> HTTP API -> RPC -> Raft -> FSM -> StateStore

Checklists

When adding new features to Nomad there are often many places to make changes. It is difficult to determine where changes must be made and easy to make mistakes.

The following checklists are meant to be copied and pasted into PRs to give developers and reviewers confidence that the proper changes have been made:

Tooling

Go tool versions