The nodes api documentation is fairly out of date, here I've updated the
entire response based on a local dev agent, rather than explicitly
adding new fields to bring us up to the current api shape.
* adds meta object to service in job spec, sends it to consul
* adds tests for service meta
* fix tests
* adds docs
* better hashing for service meta, use helper for copying meta when registering service
* tried to be DRY, but looks like it would be more work to use the
helper function
Added note to document that link-local addresses can be fingerprinted in
cases where no routable address can be found. Crosslinked to
`"fingerprint.network.disallow_link_local"` because they are somewhat
related and it is documented at a reasonable distance from this setting.
Consul Connect must route traffic between network namespaces through a
public interface (i.e. not localhost). In order to support testing in
dev mode, users needed to manually set the interface which doesn't
make for a smooth experience.
This commit adds a facility for adding optional parameters to the
`nomad agent -dev` flag and uses it to add a `-dev=connect` flag that
binds to a public interface on the host.
When rendering a task template, the `plugin` function is no longer
permitted by default and will raise an error. An operator can opt-in
to permitting this function with the new `template.function_blacklist`
field in the client configuration.
When rendering a task template, path parameters for the `file`
function will be treated as relative to the task directory by
default. Relative paths or symlinks that point outside the task
directory will raise an error. An operator can opt-out of this
protection with the new `template.disable_file_sandbox` field in the
client configuration.
The `/v1/client/fs/stream endpoint` supports tailing a file by writing
chunks out as they come in. But not all browsers support streams
(ex IE11) so we need to be able to tail a file without streaming.
The fs stream and logs endpoint use the same implementation for
filesystem streaming under the hood, but the fs stream always passes
the `follow` parameter set to true. This adds the same toggle to the
fs stream endpoint that we have for logs. It defaults to true for
backwards compatibility.
* fix navigation issue for spread guide
* skeleton for preemption guide
* background info, challenge, and pre-reqs
* steps
* rewording of intro
* re-wording
* adding more detail to intro
* clarify use of preemption in intro
- Revised Enterprise content to reflect new pricing & packaging modules
- Consolidated feature pages into single Enterprise pager
- Made each feature into its own subsection on the Enterprise page
- Added direct links to their respective guides below
- Upleveled all Enterprise features onto the navbar
- Updated redirects
Enterprise only.
Disable preemption for service and batch jobs by default.
Maintain backward compatibility in a x.y.Z release. Consider switching
the default for new clusters in the future.
This exposes a client flag to disable nomad remote exec support in
environments where access to tasks ought to be restricted.
I used `disable_remote_exec` client flag that defaults to allowing
remote exec. Opted for a client config that can be used to disable
remote exec globally, or to a subset of the cluster if necessary.
* master: (912 commits)
Update redirects.txt
Added redirect for Spark guide link
client: log when server list changes
docs: mention regression in task config validation
fix update to changelog
update CHANGELOG with datacenter config validation https://github.com/hashicorp/nomad/pull/5665
typo: "atleast" -> "at least"
implement nomad exec for rkt
docs: fixed typo
use pty/tty terminology similar to github.com/kr/pty
vendor github.com/kr/pty
drivers: implement streaming exec for executor based drivers
executors: implement streaming exec
executor: scaffolding for executor grpc handling
client: expose allocated memory per task
client improve a comment in updateNetworks
stalebot: Add 'thinking' as an exempt label (#5684)
Added Sparrow link
update links to use new canonical location
Add redirects for restructing done in GH-5667
...
Adds jrasell/sherpa to the resources page under the Integrations
section.
Replicator is no longer being maintained or has been under active
development for well over a year. I have therefore removed this
from the resources page.
- Revised "What is Nomad" copy
- Added "Key Features" section with links to task drivers & device plugins with lift-and-shift from README
- Added "Who Uses Nomad" section with users, talks, blog posts
- Removed Hadoop YARN, Docker Swarm, HTCondor from comparisons
- Revamped Guides section
- Inserted "Installing Nomad", "Upgrading", "Integrations" as persistent in Guides navbar
- Split Installing Nomad into two paths for users (one for Sandbox with "Quickstart", one for Production)
- Surfaced "Upgrading" and "Integrations" section from documentation
- Changed "Job Lifecycle" section into "Deploying & Managing Applications"
- Reworked "Operations" into "Operating Nomad"
- Reworked "Security" into "Securing Nomad"
- Segmented Namespaces, Resource Quotas, Sentinel into "Governance & Policy" subsection
- Reworked "Spark integration" into its own "Analytical Workloads" section
Add links to plugin guide because its buried under Docs -> Internal. At
least one user couldn't find it:
https://groups.google.com/d/msg/nomad-tool/5sR8MTGZFrM/8bOExpcJAgAJ
Link from:
- Devices index
- Devices community page
- Drivers index
- Drivers community page
I erred on the side of over-linking because it's just a single concise
sentence and I'd hate for people to struggle finding the guide.
Also remove some very old (4 years!) and confusing docs from the driver
index page.
- Rewrote overview description
- Reformatted "Community" scetion for consistency with GitHub
- Removed IRC (redundant with Gitter)
- Added "Community Calls" section with links to prior sessions
- Added "Webinar" section with links to prior sessions
- Revised Bug Tracker copy
- Renamed "Tools for Provisioning & Experimentation" to "How to Install Nomad" section
- Bumped up the "How to Install Nomad" section right below to second on the list "Community"
- Removed "Videos" & "Blog Posts" section in favor of accurate segmentation between user talks, webinars, blog posts
- Added "Who Uses Nomad" section in consistency with new GitHub ReadME
- Removed the "Trusted By" section and disclaimer at bottom
* navigation and initial steps of guide
* generate certs with appropriate token
* configure Nomad to use TLS
* add cli keys and certs
* add server gossip encryption section
* fix mislabeled steps
* vault paths formatting
* remove bit about cert revocation
* add clarification in challenge that we will be securing an existing Nomad cluster
* add some comments to consul-template.hcl to help user walk through it
* clarifying comments for CLI certs templates
* reorganize steps, change permissions on certs, and sub pkill command with systemctl reload nomad
* correct step reference
* add rpc upgrade mode instructions
* correct typo
Point users to security doc instead. Right now it takes a lot of
explaining to describe to users exactly how to validate the binary and
what the output of the tools used means.
For example, this is the output when validating according to the
instructions in this guide and the linked doc:
```
vagrant@linux:/tmp$ gpg --verify nomad_0.8.7_SHA256SUMS.sig
nomad_0.8.7_SHA256SUMS
gpg: Signature made Fri 11 Jan 2019 09:47:56 PM UTC using RSA key ID
348FFC4C
gpg: Good signature from "HashiCorp Security <security@hashicorp.com>"
gpg: WARNING: This key is not certified with a trusted signature!
gpg: There is no indication that the signature belongs to the
owner.
Primary key fingerprint: 91A6 E7F8 5D05 C656 30BE F189 5185 2D87 348F
FC4C
vagrant@linux:/tmp$ shasum -a 256 -c nomad_0.8.7_SHA256SUMS
shasum: ./nomad_0.8.7_darwin_amd64.zip:
./nomad_0.8.7_darwin_amd64.zip: FAILED open or read
shasum: ./nomad_0.8.7_linux_386.zip: No such file or directory
./nomad_0.8.7_linux_386.zip: FAILED open or read
shasum: ./nomad_0.8.7_linux_amd64-lxc.zip: No such file or directory
./nomad_0.8.7_linux_amd64-lxc.zip: FAILED open or read
./nomad_0.8.7_linux_amd64.zip: OK
shasum: ./nomad_0.8.7_linux_arm64.zip: No such file or directory
./nomad_0.8.7_linux_arm64.zip: FAILED open or read
shasum: ./nomad_0.8.7_linux_arm.zip: No such file or directory
./nomad_0.8.7_linux_arm.zip: FAILED open or read
shasum: ./nomad_0.8.7_windows_386.zip: No such file or directory
./nomad_0.8.7_windows_386.zip: FAILED open or read
shasum: ./nomad_0.8.7_windows_amd64.zip: No such file or directory
./nomad_0.8.7_windows_amd64.zip: FAILED open or read
shasum: WARNING: 7 listed files could not be read
```
There are only two lines that matter in all of that output:
```
...
gpg: Good signature from "HashiCorp Security <security@hashicorp.com>"
...
./nomad_0.8.7_linux_amd64.zip: OK
...
```
I feel like trying to teach users how to use and interpret these tools
in our deployment guide may be as likely to reduce confidence as
increase it.
The systemd configs spread across our repo were fairly out of sync. This
should get them on our best practices.
The deployment guide also had some strange things like running Nomad as
a non-root user. It would be fine for servers but completely breaks
clients. For simplicity I simply removed the non-root user references.
* skeleton
* configure portworx
* destroy and redeploy mysql with data intact
* rename all directories and references from persistent storage to stateful workloads
* add considerations and remove references to StorageOS
* update wording and headings
* create portworx volume externally and modify jobfile to reflect that
* fix typo
* Update website/source/guides/stateful-workloads/portworx.html.md
Co-Authored-By: Omar-Khawaja <Omar-Khawaja@users.noreply.github.com>
* Update website/source/guides/stateful-workloads/portworx.html.md
Co-Authored-By: Omar-Khawaja <Omar-Khawaja@users.noreply.github.com>
Made small adjustment to make it clear that 0.8.7 would require the legacy syntax and that the deprecation notice was more about the legacy syntax becoming unsupported at some point after v0.9.0
* update formatting and add toJSON function with explanation
* edit typo
* Update website/source/guides/operations/vault-integration/index.html.md
Co-Authored-By: Omar-Khawaja <Omar-Khawaja@users.noreply.github.com>
* fixing clarification bullet explaining the use of toJSON
* skeleton for external plugins guide section
* initial content
* add installation and configuration steps
* complete steps to deployg lxc workload
* update link
* correct typo in link
* change link name
* fix alignment and word wrap at 80
* updates
* update lxc driver download link and commands in step 3
* fix link typo
* call out pluggable drivers in task drivers section and link/add info to plugin stanza
* fix hyphenation
* removing page and nav that tells users drivers are not pluggable
* show new syntax for configuring raw_exec plugin on client
* enabled option value for raw_exec is boolean
* add plugin options section and mark client options as soon to be deprecated
* fix typos
* add plugin options for rkt task drivers and place deprecation warning in client options
* add some plugin options with plugin configuration example + mark client options as soon to be deprecated
* modify deprecation warning
* replace colon with - for options
* add docker plugin options
* update links within docker task driver to point to plugin options
* fix typo and clarify config options for lxc task driver
* replace raw_exec plugin syntax example with docker example
* create external section
* restructure lxc docs and add backward incompatibility warning
* update lxc driver doc
* add redirect for lxc driver doc
* call out plugin options and mark client config options for drivers as deprecated
* add placeholder for lxc driver binary download
* update data_dir/plugins reference with plugin_dir reference
* Update website/source/docs/external/lxc.html.md
Co-Authored-By: Omar-Khawaja <Omar-Khawaja@users.noreply.github.com>
* corrections
* remove lxc from built-in drivers navigation
* reorganize doc structure and fix redirect
* add detail about 0.9 changes
* implement suggestions/fixes
* removed extraneous punctuation
* add official lxc driver link
* initial structure
* add to affinity guide
* add to affinity guide
* update affinity guide
* spread guide
* update content
* update step 3 to spread stanza
* update
* add wording to motivate the use of spread and affinity
* improve guide description and use clearer wording to distinguish constrain from affinity
* clarify challenge to user with specific example
* improve wording in the solution section
* incorporate rest of suggestions into affinity guide
* modify spread guide description
* improve wording of spread docs
* change instance count from 5 to 10 to more easily show spread
* improving spread guide and changing demo architecture
* motivate spread a bit more
* clarification about spread
* fix wording
- docker fingerprint issues a docker api system info call to get the
list of supported OCI runtimes.
- OCI runtimes are reported as comma separated list of names
- docker driver is aware of GPU runtime presence
- docker driver throws an error when user tries to run container with
GPU, when GPU runtime is not present
- docker GPU runtime name is configurable
IOPS have been modelled as a resource since Nomad 0.1 but has never
actually been detected and there is no plan in the short term to add
detection. This is because IOPS is a bit simplistic of a unit to define
the performance requirements from the underlying storage system. In its
current state it adds unnecessary confusion and can be removed without
impacting any users. This PR leaves IOPS defined at the jobspec parsing
level and in the api/ resources since these are the two public uses of
the field. These should be considered deprecated and only exist to allow
users to stop using them during the Nomad 0.9.x release. In the future,
there should be no expectation that the field will exist.
* Add Nomad RA
* Add deployment guide and nav
* Deployment Guide update
* Minor typo fixes
* Update diagrams
* Fixes for review
* Link fixes and typo fix
* Edits following review
- Update image text from "zone" to "datacenter" to match Nomad terminology
- Clean up text based on Preetha's feedback
* Text updates
Based on feedback from Rob
* Update diagrams
* fixing spelling
* Add suggestions from Preetha and Omar
* add vault integration guide in guides section and move current vault integration content to docs section
* complete guide with image
* fix typos
* rename step 6 and fix typos
* fix typos and awkward phrasing along with links
* fix duplicated step #
* fix typo
* fix links so that pages that pointed to the original vault integration content still point there
Omar Khawaja