document artifact downloading

client/driver/executor/executor_linux.go

@@ -27,12 +27,12 @@ var (
 		"/lib":            "/lib",
 		"/lib32":          "/lib32",
 		"/lib64":          "/lib64",
+		"/run/resolvconf": "/run/resolvconf",
 		"/sbin":           "/sbin",
 		"/usr/bin":        "/usr/bin",
-		"/usr/sbin":       "/usr/sbin",
 		"/usr/lib":        "/usr/lib",
+		"/usr/sbin":       "/usr/sbin",
 		"/usr/share":      "/usr/share",
-		"/run/resolvconf": "/run/resolvconf",
 	}
 )
 

website/source/docs/drivers/docker.html.md

@@ -121,8 +121,10 @@ Example:
 ```
 task "secretservice" {
     driver = "docker"
+
     config {
         image = "secret/service"
+
         auth {
             username = "dockerhub_user"
             password = "dockerhub_password"
@@ -194,6 +196,7 @@ task "redis" {
 
     config {
       image = "redis"
+
       port_map {
         redis = 6379
       }

website/source/docs/drivers/exec.html.md

@@ -20,17 +20,10 @@ scripts or other wrappers which provide higher level features.
 
 The `exec` driver supports the following configuration in the job spec:
 
-* `command` - The command to execute. Must be provided.
+* `command` - The command to execute. Must be provided. If executing a binary
-
+  that exists on the host, the path must be absolute. If executing a binary that
-* `artifact_source` – (Optional) Source location of an executable artifact. Must
+  is download from an [`artifact`](/docs/jobspec/index.html#artifact_doc), the
-  be accessible from the Nomad client. If you specify an `artifact_source` to be
+  path can be relative from the allocations's root directory.
-  executed, you must reference it in the `command` as show in the examples below
-
-* `checksum` - (Optional) The checksum type and value for the `artifact_source`
-  image.  The format is `type:value`, where type is any of `md5`, `sha1`,
-  `sha256`, or `sha512`, and the value is the computed checksum. If a checksum
-  is supplied and does not match the downloaded artifact, the driver will fail
-  to start
 
 *   `args` - (Optional) A list of arguments to the optional `command`.
     References to environment variables or any [intepretable Nomad
@@ -41,6 +34,41 @@ The `exec` driver supports the following configuration in the job spec:
         args = ["${nomad.datacenter}", "${MY_ENV}", "${meta.foo}"]
     ```
 
+## Examples
+
+To run a binary present on the Node:
+
+```
+  task "example" {
+    driver = "exec"
+
+    config {
+      # When running a binary that exists on the host, the path must be absolute
+      command = "/bin/sleep"
+      args = ["1"]
+    }
+  }
+```
+
+To execute a binary downloaded from an [`artifact`](/docs/jobspec/index.html#artifact_doc):
+
+```
+  task "example" {
+    driver = "exec"
+
+    config {
+      command = "binary.bin"
+    }
+
+    artifact {
+      source = "https://dl.dropboxusercontent.com/u/1234/binary.bin"
+      options {
+        checksum = "sha256:abd123445ds4555555555"
+      }
+    }
+  }
+```
+
 ## Client Requirements
 
 The `exec` driver can only be run when on Linux and running Nomad as root.
@@ -48,34 +76,10 @@ The `exec` driver can only be run when on Linux and running Nomad as root.
 is only guaranteed on Linux. Further the host must have cgroups mounted properly
 in order for the driver to work.
 
-You must specify a `command` to be executed. Optionally you can specify an
-`artifact_source` to be downloaded as well. Any `command` is assumed to be present on the 
-running client, or a downloaded artifact.
-
 If you are receiving the error `* Constraint "missing drivers" filtered <> nodes`
 and using the exec driver, check to ensure that you are running Nomad as root. This
 also applies for running Nomad in -dev mode.
 
-## Examples
-
-To run a binary present on the Node:
-
-```
-  config {
-    command = "/bin/sleep"
-    args = 1
-  }
-```
-
-To execute a binary specified by `artifact_source`:
-
-```
-  config {
-    artifact_source = "https://dl.dropboxusercontent.com/u/1234/binary.bin"
-    checksum = "sha256:abd123445ds4555555555"
-    command = "binary.bin"
-  }
-```
 
 ## Client Attributes
 
@@ -96,4 +100,5 @@ resources of a process and as such the Nomad agent must be run as root.
 The chroot is populated with data in the following folders from the host
 machine:
 
-`["/bin", "/etc", "/lib", "/lib32", "/lib64", "/usr/bin", "/usr/lib", "/usr/share"]`
+`["/bin", "/etc", "/lib", "/lib32", "/lib64", "/run/resolvconf", "/sbin",
+"/usr/bin", "/usr/lib", "/usr/sbin", "/usr/share"]`

website/source/docs/drivers/java.html.md

@@ -11,21 +11,17 @@ description: |-
 Name: `java`
 
 The `Java` driver is used to execute Java applications packaged into a Java Jar
-file. The driver currently requires the Jar file be accessible via
+file. The driver requires the Jar file to be accessible from the Nomad
-HTTP from the Nomad client. 
+client via the [`artifact` downloader](/docs/jobspec/index.html#artifact_doc). 
 
 ## Task Configuration
 
 The `java` driver supports the following configuration in the job spec:
 
-* `artifact_source` - The hosted location of the source Jar file. Must be
+* `jar_path` - The path to the downloaded Jar. In most cases this will just be
-  accessible from the Nomad client
+  the name of the Jar. However, if the supplied artifact is an archive that
-
+  contains the Jar in a subfolder, the path will need to be the relative path
-* `checksum` - (Optional) The checksum type and value for the `artifact_source`
+  (`subdir/from_archive/my.jar`).
-  image.  The format is `type:value`, where type is any of `md5`, `sha1`,
-  `sha256`, or `sha512`, and the value is the computed checksum. If a checksum
-  is supplied and does not match the downloaded artifact, the driver will fail
-  to start
 
 *   `args` - (Optional) A list of arguments to the optional `command`.
     References to environment variables or any [intepretable Nomad
@@ -39,30 +35,36 @@ The `java` driver supports the following configuration in the job spec:
 * `jvm_options` - (Optional) A list of JVM options to be passed while invoking
   java. These options are passed not validated in any way in Nomad.
 
-## Client Requirements
-
-The `java` driver requires Java to be installed and in your systems `$PATH`.
-The `artifact_source` must be accessible by the node running Nomad. This can be an
-internal source, private to your cluster, but it must be reachable by the client 
-over HTTP. 
-
 ## Examples
 
 A simple config block to run a Java Jar:
 
-```json
+```
-# Define a task to run
 task "web" {
-  # Run a Java Jar
   driver = "java"
 
   config {
-    artifact_source = "https://dl.dropboxusercontent.com/u/1234/hello.jar"
+    jar_path = "hello.jar"
-    checksum = "md5:123445555555555"
     jvm_options = "-Xmx2048m -Xms256m"
   }
+
+  # Specifying an artifact is required with the "java"
+  # driver. This is the # mechanism to ship the Jar to be run.
+  artifact {
+    source = "https://dl.dropboxusercontent.com/u/1234/hello.jar"
+
+    options {
+      checksum = "md5:123445555555555"
+    }
+  }
 ```
 
+## Client Requirements
+
+The `java` driver requires Java to be installed and in your systems `$PATH`. The
+task must also specify at least one artifact to download as this is the only way
+to retrieve the Jar being run.
+
 ## Client Attributes
 
 The `java` driver will set the following client attributes:

website/source/docs/drivers/qemu.html.md

@@ -19,17 +19,17 @@ resource allocation.
 The `Qemu` driver can execute any regular `qemu` image (e.g. `qcow`, `img`,
 `iso`), and is currently invoked with `qemu-system-x86_64`.
 
+The driver requires the image to be accessible from the Nomad client via the
+[`artifact` downloader](/docs/jobspec/index.html#artifact_doc). 
+
 ## Task Configuration
 
 The `Qemu` driver supports the following configuration in the job spec:
 
-* `artifact_source` - The hosted location of the source Qemu image. Must be accessible
+* `image_path` - The path to the downloaded image. In most cases this will just be
-  from the Nomad client, via HTTP.
+  the name of the image. However, if the supplied artifact is an archive that
-
+  contains the image in a subfolder, the path will need to be the relative path
-* `checksum` - (Optional) The checksum type and value for the `artifact_source` image.
+  (`subdir/from_archive/my.img`).
-  The format is `type:value`, where type is any of `md5`, `sha1`, `sha256`, or `sha512`,
-  and the value is the computed checksum. If a checksum is supplied and does not
-  match the downloaded artifact, the driver will fail to start
 
 * `accelerator` - (Optional) The type of accelerator to use in the invocation.
   If the host machine has `Qemu` installed with KVM support, users can specify
@@ -40,12 +40,35 @@ The `Qemu` driver supports the following configuration in the job spec:
   `port_map { db = 6539 }` would forward the host port with label `db` to the
   guest vm's port 6539.
 
+## Examples
+
+A simple config block to run a `Qemu` image:
+
+```
+task "virtual" {
+  driver = "qemu"
+
+  config {
+    image_path = "linux.img"
+    accelerator = "kvm"
+  }
+
+  # Specifying an artifact is required with the "qemu"
+  # driver. This is the # mechanism to ship the image to be run.
+  artifact {
+    source = "https://dl.dropboxusercontent.com/u/1234/linux.img"
+
+    options {
+      checksum = "md5:123445555555555"
+    }
+  }
+```
+
 ## Client Requirements
 
 The `Qemu` driver requires Qemu to be installed and in your system's `$PATH`.
-The `artifact_source` must be accessible by the node running Nomad. This can be an
+The task must also specify at least one artifact to download as this is the only
-internal source, private to your cluster, but it must be reachable by the client
+way to retrieve the image being run.
-over HTTP.
 
 ## Client Attributes
 

website/source/docs/drivers/raw_exec.html.md

@@ -18,17 +18,10 @@ As such, it should be used with extreme care and is disabled by default.
 
 The `raw_exec` driver supports the following configuration in the job spec:
 
-* `command` - The command to execute. Must be provided.
+* `command` - The command to execute. Must be provided. If executing a binary
-
+  that exists on the host, the path must be absolute. If executing a binary that
-* `artifact_source` – (Optional) Source location of an executable artifact. Must
+  is download from an [`artifact`](/docs/jobspec/index.html#artifact_doc), the
-  be accessible from the Nomad client. If you specify an `artifact_source` to be
+  path can be relative from the allocations's root directory.
-  executed, you must reference it in the `command` as show in the examples below
-
-* `checksum` - (Optional) The checksum type and value for the `artifact_source`
-  image.  The format is `type:value`, where type is any of `md5`, `sha1`,
-  `sha256`, or `sha512`, and the value is the computed checksum. If a checksum
-  is supplied and does not match the downloaded artifact, the driver will fail
-  to start
 
 *   `args` - (Optional) A list of arguments to the optional `command`.
     References to environment variables or any [intepretable Nomad
@@ -39,6 +32,41 @@ The `raw_exec` driver supports the following configuration in the job spec:
         args = ["${nomad.datacenter}", "${MY_ENV}", "${meta.foo}"]
     ```
 
+## Examples
+
+To run a binary present on the Node:
+
+```
+  task "example" {
+    driver = "raw_exec"
+
+    config {
+      # When running a binary that exists on the host, the path must be absolute
+      command = "/bin/sleep"
+      args = ["1"]
+    }
+  }
+```
+
+To execute a binary downloaded from an [`artifact`](/docs/jobspec/index.html#artifact_doc):
+
+```
+  task "example" {
+    driver = "raw_exec"
+
+    config {
+      command = "binary.bin"
+    }
+
+    artifact {
+      source = "https://dl.dropboxusercontent.com/u/1234/binary.bin"
+      options {
+        checksum = "sha256:abd123445ds4555555555"
+      }
+    }
+  }
+```
+
 ## Client Requirements
 
 The `raw_exec` driver can run on all supported operating systems. It is however
@@ -54,31 +82,6 @@ explicitly enable the `raw_exec` driver in the client's
   }
 ```
 
-You must specify a `command` to be executed. Optionally you can specify an
-`artifact_source` to be executed. Any `command` is assumed to be present on the 
-running client, or a downloaded artifact
-
-## Examples
-
-To run a binary present on the Node:
-
-```
-  config {
-    command = "/bin/sleep"
-    args = 1
-  }
-```
-
-To execute a binary specified by `artifact_source`:
-
-```
-  config {
-    artifact_source = "https://dl.dropboxusercontent.com/u/1234/binary.bin"
-    checksum = "sha256:133jifjiofu9090fsadjofsdjlk"
-    command = "binary.bin"
-  }
-```
-
 ## Client Attributes
 
 The `raw_exec` driver will set the following client attributes:

website/source/docs/jobspec/index.html.md

@@ -411,6 +411,8 @@ would be required for the task would be 60MB.
 
 ### Artifact
 
+<a id="artifact_doc"></a>
+
 Nomad downloads artifacts using
 [`go-getter`](https://github.com/hashicorp/go-getter). The `go-getter` library
 allows downloading of artifacts from various sources using a URL as the input
@@ -443,6 +445,20 @@ options {
 }
 ```
 
+An example of downloading and unzipping an archive is as simple as:
+
+```
+artifact {
+  # The archive will be extracted before the task is run, making 
+  # it easy to ship configurations with your binary.
+  source = "https://example.com/my.zip"
+
+  options {
+    checksum = "md5:7f4b3e3b4dd5150d4e5aaaa5efada4c3"
+  }
+}
+```
+
 ## JSON Syntax
 
 Job files can also be specified in JSON. The conversion is straightforward

website/source/docs/upgrade/upgrade-specific.html.md

@@ -15,6 +15,14 @@ details provided for their upgrades as a result of new features or changed
 behavior. This page is used to document those details separately from the
 standard upgrade flow.
 
+## Nomad 0.3.1
+
+Nomad 0.3.1 removes artifact downloading from driver configs and places them as
+a first class element of the task. As such, jobs will have to be rewritten in
+the proper format and resubmitted to Nomad. Nomad clients will properly
+re-attach to existing tasks but job definitions must be updated before they can
+be dispatched to clients running 0.3.1.
+
 ## Nomad 0.3.0
 
 Nomad 0.3.0 has made several substantial changes to job files included a new