## 0.12.1 (Unreleased) BUG FIXES: * jobspec: Fixed validation of multi-region datacenters to allow empty region `datacenters` to default to job-level `datacenters`. [[GH-8426](https://github.com/hashicorp/nomad/issues/8426)] * ui: Fixed order of column headers in client allocations table. [[GH-8409](https://github.com/hashicorp/nomad/pull/8409)] * ui: Fixed stale namespaces after changing acl tokens. [[GH-8413](https://github.com/hashicorp/nomad/issues/8413)] * ui: Fixed missing namespace query param after changing acl tokens [[GH-8413](https://github.com/hashicorp/nomad/issues/8413)] * ui: Fixed runtime error when clicking "Run Job" while a prefix filter is set [[GH-8412](https://github.com/hashicorp/nomad/issues/8412)] * vault: Fixed a bug where vault identity policies not considered in permissions check [[GH-7732](https://github.com/hashicorp/nomad/issues/7732)] ## 0.12.0 (July 9, 2020) FEATURES: * **Preemption**: Preemption is now an open source feature * **Licensing (Enterprise)**: Nomad Enterprise now requires a license [[GH-8076](https://github.com/hashicorp/nomad/issues/8076)] * **Multiregion Deployments (Enterprise)**: Nomad Enterprise now enables orchestrating deployments across multiple regions. [[GH-8184](https://github.com/hashicorp/nomad/issues/8184)] * **Snapshot Backup and Restore**: Nomad eases disaster recovery with new endpoints and commands for point-in-time snapshots. * **Debug Log Archive**: Nomad debug captures state and logs to facilitate support [[GH-8273](https://github.com/hashicorp/nomad/issues/8273)] * **Container Network Interface (CNI)**: Support for third-party vendors using the CNI plugin system. [[GH-7518](https://github.com/hashicorp/nomad/issues/7518)] * **Multi-interface Networking**: Support for scheduling on specific network interfaces. [[GH-8208](https://github.com/hashicorp/nomad/issues/8208)] * **Consul Connect Native**: Support for running Consul Connect Native tasks. [[GH-6083](https://github.com/hashicorp/nomad/issues/6083)] * **Global Search**: Access jobs and clients from anywhere in the UI using the always available global search bar. [[GH-8175](https://github.com/hashicorp/nomad/issues/8175)] * **Monitor UI**: Stream client and agent logs from the UI just like you would with the nomad monitor CLI command. [[GH-8177](https://github.com/hashicorp/nomad/issues/8177)] * **Scaling UI**: Quickly adjust the count of a task group from the UI for task groups with a scaling declaration. [[GH-8207](https://github.com/hashicorp/nomad/issues/8207)] __BACKWARDS INCOMPATIBILITIES:__ * driver/docker: The Docker driver no longer allows binding host volumes by default. Operators can set `volume` `enabled` plugin configuration to restore previous permissive behavior. [[GH-8261](https://github.com/hashicorp/nomad/issues/8261)] * driver/qemu: The Qemu driver requires images to reside in a operator-defined paths allowed for task access. [[GH-8261](https://github.com/hashicorp/nomad/issues/8261)] IMPROVEMENTS: * core: Support for persisting previous task group counts when updating a job [[GH-8168](https://github.com/hashicorp/nomad/issues/8168)] * core: Block Job.Scale actions when the job is under active deployment [[GH-8187](https://github.com/hashicorp/nomad/issues/8187)] * api: Better error messages around Scaling->Max [[GH-8360](https://github.com/hashicorp/nomad/issues/8360)] * api: Persist previous count with scaling events [[GH-8167](https://github.com/hashicorp/nomad/issues/8167)] * api: Support querying for jobs and allocations across all namespaces [[GH-8192](https://github.com/hashicorp/nomad/issues/8192)] * api: New `/agent/host` endpoint returns diagnostic information about the host [[GH-8325](https://github.com/hashicorp/nomad/pull/8325)] * build: Updated to Go 1.14.4 [[GH-8172](https://github.com/hashicorp/nomad/issues/9172)] * build: Switched to Go modules for dependency management [[GH-8041](https://github.com/hashicorp/nomad/pull/8041)] * connect: Infer service task parameter where possible [[GH-8274](https://github.com/hashicorp/nomad/issues/8274)] * server: Added `raft_multiplier` config to tweak Raft related timeouts [[GH-8082](https://github.com/hashicorp/nomad/issues/8082)] BUG FIXES: * cli: Fixed malformed alloc status address list when listing more than 1 address [[GH-8161](https://github.com/hashicorp/nomad/issues/8161)] * client: Fixed a bug where stdout/stderr were not properly reopened for community task drivers [[GH-8155](https://github.com/hashicorp/nomad/issues/8155)] * client: Fixed a bug where batch job sidecars may be left running after the main task completes [[GH-8311](https://github.com/hashicorp/nomad/issues/8311)] * connect: Fixed a bug where custom `sidecar_task` definitions were being shared [[GH-8337](https://github.com/hashicorp/nomad/issues/8337)] * csi: Fixed a bug where `NodeStageVolume` and `NodePublishVolume` requests were not receiving volume context [[GH-8239](https://github.com/hashicorp/nomad/issues/8239)] * driver/docker: Fixed a bug to set correct value for `memory-swap` when using `memory_hard_limit` [[GH-8153](https://github.com/hashicorp/nomad/issues/8153)] * ui: The log streamer will now always follow logs when the current scroll position is the end of the buffer. [[GH-8177](https://github.com/hashicorp/nomad/issues/8177)] * ui: The task group detail page no longer makes excessive requests to the allocation and stats endpoints. [[GH-8216](https://github.com/hashicorp/nomad/issues/8216)] * ui: Polling endpoints that have yet to be fetched normally works as expected (regression from 0.11.3). [[GH-8207](https://github.com/hashicorp/nomad/issues/8207)] ## 0.11.3 (June 5, 2020) IMPROVEMENTS: * build: Updated to Go 1.14.3 [[GH-7431](https://github.com/hashicorp/nomad/issues/7970)] * csi: Return better error messages [[GH-7984](https://github.com/hashicorp/nomad/issues/7984)] [[GH-8030](https://github.com/hashicorp/nomad/issues/8030)] * csi: Move volume claim releases out of evaluation workers [[GH-8021](https://github.com/hashicorp/nomad/issues/8021)] * csi: Added support for `VolumeContext` and `VolumeParameters` [[GH-7957](https://github.com/hashicorp/nomad/issues/7957)] * driver/docker: Added support for `memory_hard_limit` configuration in docker task driver [[GH-2093](https://github.com/hashicorp/nomad/issues/2093)] * logging: Remove spurious error log on task shutdown [[GH-8028](https://github.com/hashicorp/nomad/issues/8028)] * ui: Added filesystem browsing for allocations [[GH-5871](https://github.com/hashicorp/nomad/pull/7951)] BUG FIXES: * core: Fixed a critical bug causing agent to become unresponsive [[GH-7431](https://github.com/hashicorp/nomad/issues/7970)], [[GH-8163](https://github.com/hashicorp/nomad/issues/8163)] * core: Fixed a bug impacting performance of scheduler on a server after it steps down [[GH-8089](https://github.com/hashicorp/nomad/issues/8089)] * core: Fixed a bug where new leader may take a long time until it can process requests [[GH-8036](https://github.com/hashicorp/nomad/issues/8036)] * core: Fixed a bug where stop_after_client_disconnect could cause the server to become unresponsive [[GH-8098](https://github.com/hashicorp/nomad/issues/8098) * core: Fixed a bug where an internal metadata, ClusterID, may not be initialized properly upon a slow server upgrade [[GH-8078](https://github.com/hashicorp/nomad/issues/8078)] * api: Fixed a bug where setting connect sidecar task resources could fail [[GH-7993](https://github.com/hashicorp/nomad/issues/7993)] * client: Fixed a bug where artifact downloads failed on redirects [[GH-7854](https://github.com/hashicorp/nomad/issues/7854)] * csi: Validate empty volume arguments in API. [[GH-8027](https://github.com/hashicorp/nomad/issues/8027)] ## 0.11.2 (May 14, 2020) FEATURES: * **Task dependencies UI**: task lifecycle charts and details IMPROVEMENTS: * core: Added support for a per-group policy to stop tasks when a client is disconnected [[GH-2185](https://github.com/hashicorp/nomad/issues/2185)] * core: Allow spreading allocations as an alternative to binpacking [[GH-7810](https://github.com/hashicorp/nomad/issues/7810)] * client: Improve AWS CPU performance fingerprinting [[GH-7681](https://github.com/hashicorp/nomad/issues/7681)] * csi: Added support for volume secrets [[GH-7923](https://github.com/hashicorp/nomad/issues/7923)] * csi: Added periodic garbage collection of plugins and volume claims [[GH-7825](https://github.com/hashicorp/nomad/issues/7825)] * csi: Improved performance of volume claim releases by moving work out of scheduler [[GH-7794](https://github.com/hashicorp/nomad/issues/7794)] * driver/docker: Added support for custom runtimes [[GH-7932](https://github.com/hashicorp/nomad/pull/7932)] * ui: Added ACL-checking to conditionally turn off exec button [[GH-7919](https://github.com/hashicorp/nomad/pull/7919)] * ui: CSI searchable volumes and plugins pages [[GH-7895](https://github.com/hashicorp/nomad/issues/7895)] * ui: CSI plugins list and etail pages [[GH-7872](https://github.com/hashicorp/nomad/issues/7872)] [[GH-7911](https://github.com/hashicorp/nomad/issues/7911)] * ui: CSI volume constraints table [[GH-7872](https://github.com/hashicorp/nomad/issues/7872)] BUG FIXES: * core: job scale status endpoint was returning incorrect counts [[GH-7789](https://github.com/hashicorp/nomad/issues/7789)] * core: Fixed bugs related to periodic jobs scheduled during daylight saving transition periods [[GH-7894](https://github.com/hashicorp/nomad/issues/7894)] * core: Fixed a bug where scores for allocations were biased toward nodes with resource reservations [[GH-7730](https://github.com/hashicorp/nomad/issues/7730)] * api: api.ScalingEvent struct was missing .Count [[GH-7915](https://github.com/hashicorp/nomad/pulls/7915)] * api: validate scale count value is not negative [[GH-7902](https://github.com/hashicorp/nomad/issues/7902)] * api: autoscaling policies should not be returned for stopped jobs [[GH-7768](https://github.com/hashicorp/nomad/issues/7768)] * client: Fixed a bug where an multi-task allocation maybe considered unhealthy if some tasks are slow to start [[GH-7944](https://github.com/hashicorp/nomad/issues/7944)] * csi: Fixed checking of volume validation responses from plugins [[GH-7831](https://github.com/hashicorp/nomad/issues/7831)] * csi: Fixed counting of healthy and expected plugins after plugin job updates or stops [[GH-7844](https://github.com/hashicorp/nomad/issues/7844)] * csi: Added checkpointing to volume claim release to avoid unreleased claims on plugin errors [[GH-7782](https://github.com/hashicorp/nomad/issues/7782)] * driver/docker: Fixed a bug preventing garbage collecting unused docker images [[GH-7947](https://github.com/hashicorp/nomad/issues/7947)] * jobspec: autoscaling policy block should return a parsing error multiple `policy` blocks are provided [[GH-7716](https://github.com/hashicorp/nomad/issues/7716)] * ui: Fixed a bug where exec popup had incorrect URL for jobs where name ≠ id [[GH-7814](https://github.com/hashicorp/nomad/issues/7814)] * ui: Fixed a timeout issue where if the log stream request to a client eventually returns but only after the timeout it never gets closed [[GH-7820](https://github.com/hashicorp/nomad/issues/7820)] * ui: Setting a namespace on Volumes or Jobs persists that namespace choice when switching to another namespace-away page [[GH-7896](https://github.com/hashicorp/nomad/issues/7896)] * ui: Fixed a bug where clicking stdout or stderr when already on that clicked view would pause log streaming [[GH-7820](https://github.com/hashicorp/nomad/issues/7820)] * ui: Fixed a race condition that made swithing from stdout to stderr too quickly show an error [[GH-7820](https://github.com/hashicorp/nomad/issues/7820)] * ui: Switching namespaces now redirects to Volumes instead of Jobs when on a Storage page [[GH-7896](https://github.com/hashicorp/nomad/issues/7896)] * ui: Allocations always report resource reservations based on thier own job version [[GH-7855](https://github.com/hashicorp/nomad/issues/7855)] * vault: Fixed a bug where nomad retries revoking tokens indefinitely [[GH-7959](https://github.com/hashicorp/nomad/issues/7959)] ## 0.11.1 (April 22, 2020) BUG FIXES: * core: Fixed a bug that only ran a task `shutdown_delay` if the task had a registered service [[GH-7663](https://github.com/hashicorp/nomad/issues/7663)] * core: Fixed a panic when garbage collecting a job with allocations spanning multiple versions [[GH-7758](https://github.com/hashicorp/nomad/issues/7758)] * agent: Fixed a bug where http server logs did not honor json log formatting, and reduced http server logging level to Trace [[GH-7748](https://github.com/hashicorp/nomad/issues/7748)] * connect: Fixed bugs where some connect parameters would be ignored [[GH-7690](https://github.com/hashicorp/nomad/pull/7690)] [[GH-7684](https://github.com/hashicorp/nomad/pull/7684)] * connect: Fixed a bug where an absent connect sidecar_service stanza would trigger panic [[GH-7683](https://github.com/hashicorp/nomad/pull/7683)] * connect: Fixed a bug where some connect proxy fields would be dropped from 'job inspect' output [[GH-7397](https://github.com/hashicorp/nomad/issues/7397)] * csi: Fixed a panic when claiming a volume for an allocation that was already garbage collected [[GH-7760](https://github.com/hashicorp/nomad/issues/7760)] * csi: Fixed a bug where CSI plugins with `NODE_STAGE_VOLUME` capabilities were receiving an incorrect volume ID [[GH-7754](https://github.com/hashicorp/nomad/issues/7754)] * driver/docker: Fixed a bug where retrying failed docker creation may in rare cases trigger a panic [[GH-7749](https://github.com/hashicorp/nomad/issues/7749)] * scheduler: Fixed a bug in managing allocated devices for a job allocation in in-place update scenarios [[GH-7762](https://github.com/hashicorp/nomad/issues/7762)] * vault: Upgrade http2 library to fix Vault API calls that fail with `http2: no cached connection was available` [[GH-7673](https://github.com/hashicorp/nomad/issues/7673)] ## 0.11.0 (April 8, 2020) FEATURES: * **Container Storage Interface [beta]**: Nomad has expanded support of stateful workloads through support for CSI plugins. * **Exec UI**: an in-browser terminal for connecting to running allocations. * **Audit Logging (Enterprise)**: Audit logging support for Nomad Enterprise. * **Scaling APIs**: new scaling policy API and job scaling APIs to support external autoscalers * **Task Dependencies**: introduces `lifecycle` stanza with prestart and sidecar hooks for tasks within a task group __BACKWARDS INCOMPATIBILITIES:__ * driver/rkt: The Rkt driver is no longer packaged with Nomad and is instead distributed separately as a driver plugin. Further, the Rkt driver codebase is now in a separate [repository](https://github.com/hashicorp/nomad-driver-rkt). IMPROVEMENTS: * core: Optimized streaming RPCs made between Nomad agents [[GH-7044](https://github.com/hashicorp/nomad/issues/7044)] * build: Updated to Go 1.14.1 [[GH-7431](https://github.com/hashicorp/nomad/issues/7431)] * consul: Added support for configuring `enable_tag_override` on service stanzas. [[GH-2057](https://github.com/hashicorp/nomad/issues/2057)] * client: Updated consul-template library to v0.24.1 - added support for working with consul connect. [Deprecated vault_grace](https://nomadproject.io/guides/upgrade/upgrade-specific/#nomad-0110) [[GH-7170](https://github.com/hashicorp/nomad/pull/7170)] * driver/exec: Added `no_pivot_root` option for ramdisk use [[GH-7149](https://github.com/hashicorp/nomad/issues/7149)] * jobspec: Added task environment interpolation to `volume_mount` [[GH-7364](https://github.com/hashicorp/nomad/issues/7364)] * jobspec: Added support for a per-task restart policy [[GH-7288](https://github.com/hashicorp/nomad/pull/7288)] * server: Added minimum quorum check to Autopilot with minQuorum option [[GH-7171](https://github.com/hashicorp/nomad/issues/7171)] * connect: Added support for specifying Envoy expose path configurations [[GH-7323](https://github.com/hashicorp/nomad/pull/7323)] [[GH-7396](https://github.com/hashicorp/nomad/pull/7515)] * connect: Added support for using Connect with TLS enabled Consul agents [[GH-7602](https://github.com/hashicorp/nomad/pull/7602)] BUG FIXES: * core: Fixed a bug where group network mode changes were not honored [[GH-7414](https://github.com/hashicorp/nomad/issues/7414)] * core: Optimized and fixed few bugs in underlying RPC handling [[GH-7044](https://github.com/hashicorp/nomad/issues/7044)] [[GH-7045](https://github.com/hashicorp/nomad/issues/7045)] * api: Fixed a panic when canonicalizing a jobspec with an incorrect job type [[GH-7207](https://github.com/hashicorp/nomad/pull/7207)] * api: Fixed a bug where calling the node GC or GcAlloc endpoints resulted in an error EOF return on successful requests [[GH-5970](https://github.com/hashicorp/nomad/issues/5970)] * api: Fixed a bug where `/client/allocations/...` (e.g. allocation stats) requests may hang in special cases after a leader election [[GH-7370](https://github.com/hashicorp/nomad/issues/7370)] * cli: Fixed a bug where `nomad agent -dev` fails on Windows [[GH-7534](https://github.com/hashicorp/nomad/pull/7534)] * cli: Fixed a panic when displaying device plugins without stats [[GH-7231](https://github.com/hashicorp/nomad/issues/7231)] * cli: Fixed a bug where `alloc exec` command in TLS environments may fail [[GH-7274](https://github.com/hashicorp/nomad/issues/7274)] * client: Fixed a panic when running in Debian with `/etc/debian_version` is empty [[GH-7350](https://github.com/hashicorp/nomad/issues/7350)] * client: Fixed a bug affecting network detection in environments that mimic the EC2 Metadata API [[GH-7509](https://github.com/hashicorp/nomad/issues/7509)] * client: Fixed a bug where a multi-task allocation maybe considered healthy despite a task restarting [[GH-7383](https://github.com/hashicorp/nomad/issues/7383)] * consul: Fixed a bug where modified Consul service definitions would not be updated [[GH-6459](https://github.com/hashicorp/nomad/issues/6459)] * connect: Fixed a bug where Connect enabled allocation would not stop after promotion [[GH-7540](https://github.com/hashicorp/nomad/issues/7540)] * connect: Fixed a bug where restarting a client would prevent Connect enabled allocations from cleaning up properly [[GH-7643](https://github.com/hashicorp/nomad/issues/7643)] * driver/docker: Fixed handling of seccomp `security_opts` option [[GH-7554](https://github.com/hashicorp/nomad/issues/7554)] * driver/docker: Fixed a bug causing docker containers to use swap memory unexpectedly [[GH-7550](https://github.com/hashicorp/nomad/issues/7550)] * scheduler: Fixed a bug where changes to task group `shutdown_delay` were not persisted or displayed in plan output [[GH-7618](https://github.com/hashicorp/nomad/issues/7618)] * ui: Fixed handling of multi-byte unicode characters in allocation log view [[GH-7470](https://github.com/hashicorp/nomad/issues/7470)] [[GH-7551](https://github.com/hashicorp/nomad/pull/7551)] ## 0.10.5 (March 24, 2020) SECURITY: * server: Override content-type headers for unsafe content. CVE-2020-10944 [[GH-7468](https://github.com/hashicorp/nomad/issues/7468)] ## 0.10.4 (February 19, 2020) FEATURES: * api: Nomad now supports ability to remotely request /debug/pprof endpoints from a remote agent. [[GH-6841](https://github.com/hashicorp/nomad/issues/6841)] * consul/connect: Nomad may now register Consul Connect services when Consul is configured with ACLs enabled [[GH-6701](https://github.com/hashicorp/nomad/issues/6701)] * jobspec: Add `shutdown_delay` to task groups so task groups can delay shutdown after deregistering from Consul [[GH-6746](https://github.com/hashicorp/nomad/issues/6746)] IMPROVEMENTS: * Our Windows 32-bit and 64-bit executables for this version and up will be signed with a HashiCorp cert. Windows users will no longer see a warning about an "unknown publisher" when running our software. * build: Updated to Go 1.12.16 [[GH-7009](https://github.com/hashicorp/nomad/issues/7009)] * cli: Included namespace in output when querying job status [[GH-6912](https://github.com/hashicorp/nomad/issues/6912)] * cli: Added option to change the name of the file created by the `nomad init` command [[GH-6520]](https://github.com/hashicorp/nomad/pull/6520) * client: Supported AWS EC2 Instance Metadata Service Version 2 (IMDSv2) [[GH-6779](https://github.com/hashicorp/nomad/issues/6779)] * consul: Add support for service `canary_meta` [[GH-6690](https://github.com/hashicorp/nomad/pull/6690)] * driver/docker: Added a `disable_log_collection` parameter to disable nomad log collection [[GH-6820](https://github.com/hashicorp/nomad/issues/6820)] * server: Introduced a `default_scheduler_config` config parameter to seed initial preemption configuration. [[GH-6935](https://github.com/hashicorp/nomad/issues/6935)] * scheduler: Removed penalty for allocation's previous node if the allocation did not fail. [[GH-6781](https://github.com/hashicorp/nomad/issues/6781)] * scheduler: Reduced logging verbosity during preemption [[GH-6849](https://github.com/hashicorp/nomad/issues/6849)] * ui: Updated Run Job button to be conditionally enabled according to ACLs [[GH-5944](https://github.com/hashicorp/nomad/pull/5944)] BUG FIXES: * agent: Fixed a panic when using `nomad monitor` on a client node [[GH-7053](https://github.com/hashicorp/nomad/issues/7053)] * agent: Fixed race condition in logging when using `nomad monitor` command [[GH-6872](https://github.com/hashicorp/nomad/issues/6872)] * agent: Fixed a bug where `nomad monitor -server-id` only work for a server's name instead of uuid or name [[GH-7015](https://github.com/hashicorp/nomad/issues/7015)] * core: Addressed an inconsistency where allocations created prior to 0.9 had missing fields [[GH-6922](https://github.com/hashicorp/nomad/issues/6922)] * cli: Fixed a bug where error messages appeared interleaved with help text inconsistently [[GH-6865](https://github.com/hashicorp/nomad/issues/6865)] * cli: Fixed a bug where `nomad monitor -node-id` would cause a cli panic when no nodes where found [[GH-6828](https://github.com/hashicorp/nomad/issues/6828)] * config: Fixed a bug where agent startup would fail if the `consul.timeout` configuration was set [[GH-6907](https://github.com/hashicorp/nomad/issues/6907)] * consul: Fixed a bug where script-based health checks would fail if the service configuration included interpolation [[GH-6916](https://github.com/hashicorp/nomad/issues/6916)] * consul/connect: Fixed a bug where Connect-enabled jobs failed to validate when service names used interpolation [[GH-6855](https://github.com/hashicorp/nomad/issues/6855)] * drivers: Fixed a bug where exec, java, and raw_exec drivers collected and emited stats every second regardless of the telemetry config [[GH-7043](https://github.com/hashicorp/nomad/issues/7043)] * driver/exec: Fixed a bug where systemd cgroup wasn't removed upon a task completion [[GH-6839](https://github.com/hashicorp/nomad/issues/6839)] * server: Fixed a deadlock that may occur when server leadership flaps very quickly [[GH-6977](https://github.com/hashicorp/nomad/issues/6977)] * scheduler: Fixed a bug that caused evicted allocs on a lost node to be stuck in running [[GH-6902](https://github.com/hashicorp/nomad/issues/6902)] * scheduler: Fixed a bug where `nomad job plan/apply` returned errors instead of ignoring system job updates for ineligible nodes. [[GH-6996](https://github.com/hashicorp/nomad/issues/6996)] * scheduler: Fixed a bug where canary allocations where not properly stored across servers during deployments [[GH-6975](https://github.com/hashicorp/nomad/pull/6975)] SECURITY: * client: Nomad will no longer pass through the `CONSUL_HTTP_TOKEN` environment variable when launching a task. [[GH-7131](https://github.com/hashicorp/nomad/issues/7131)] ## 0.10.3 (January 29, 2020) SECURITY: * agent: Added unauthenticated connection timeouts and limits to prevent resource exhaustion. CVE-2020-7218 [[GH-7002](https://github.com/hashicorp/nomad/issues/7002)] * server: Fixed insufficient validation for role and region for RPC connections when TLS enabled. CVE-2020-7956 [[GH-7003](https://github.com/hashicorp/nomad/issues/7003)] IMPROVEMENTS: * build: Updated to Go 1.12.16 ## 0.10.2 (December 4, 2019) NOTES: * cli: Our [nomad_0.10.2_darwin_amd64_notarized](https://releases.hashicorp.com/nomad/0.10.2/nomad_0.10.2_darwin_amd64_notarized.zip) release has been signed and notarized according to Apple's requirements. In the future, darwin releases will be signed and notarized with our standard naming convention. Prior to this release, MacOS 10.15+ users attempting to run our software may see the error: "'nomad' cannot be opened because the developer cannot be verified." This error affected all MacOS 10.15+ users who downloaded our software directly via web browsers, and was caused by [changes to Apple's third-party software requirements](https://developer.apple.com/news/?id=04102019a). MacOS 10.15+ users should plan to upgrade to 0.10.2+. FEATURES: * **Nomad Monitor**: New `nomad monitor` command allows remotely following the logs of any Nomad Agent (clients or servers). See https://nomadproject.io/docs/commands/monitor.html * **Docker Container Cleanup**: Nomad will now automatically remove Docker containers for tasks leaked due to Nomad or Docker crashes or bugs. IMPROVEMENTS: * agent: Added support for running under Windows Service Manager [[GH-6220](https://github.com/hashicorp/nomad/issues/6220)] * api: Added `StartedAt` field to `Node.DrainStrategy` [[GH-6698](https://github.com/hashicorp/nomad/issues/6698)] * api: Added JSON representation of rules to policy endpoint response [[GH-6017](https://github.com/hashicorp/nomad/pull/6017)] * api: Update policy endpoint to permit anonymous access [[GH-6021](https://github.com/hashicorp/nomad/issues/6021)] * build: Updated to Go 1.12.13 [[GH-6606](https://github.com/hashicorp/nomad/issues/6606)] * cli: Show full ID in node and alloc individual status views [[GH-6425](https://github.com/hashicorp/nomad/issues/6425)] * client: Enable setting tags on Consul Connect sidecar service [[GH-6448](https://github.com/hashicorp/nomad/issues/6448)] * client: Added support for downloading artifacts from Google Cloud Storage [[GH-6692](https://github.com/hashicorp/nomad/pull/6692)] * command: Added -tls-server-name flag [[GH-6370](https://github.com/hashicorp/nomad/issues/6370)] * command: Added `nomad monitor` command to stream logs at a specified level for debugging [[GH-6499](https://github.com/hashicorp/nomad/issues/6499)] * quota: Added support for network bandwidth quota limits in Nomad enterprise BUG FIXES: * core: Ignore `server` config values if `server` is disabled [[GH-6047](https://github.com/hashicorp/nomad/issues/6047)] * core: Added `semver` constraint for strict Semver 2.0 version comparisons [[GH-6699](https://github.com/hashicorp/nomad/issues/6699)] * core: Fixed server panic caused by a plan evicting and preempting allocs on a node [[GH-6792](https://github.com/hashicorp/nomad/issues/6792)] * api: Return a 404 if endpoint not found instead of redirecting to /ui/ [[GH-6658](https://github.com/hashicorp/nomad/issues/6658)] * api: Decompress web socket response body if gzipped on error responses [[GH-6650](https://github.com/hashicorp/nomad/issues/6650)] * api: Fixed a bug where some FS/Allocation API endpoints didn't return error messages [[GH-6427](https://github.com/hashicorp/nomad/issues/6427)] * api: Return 40X status code for failing ACL requests, rather than 500 [[GH-6421](https://github.com/hashicorp/nomad/issues/6421)] * cli: Made scoring column orders consistent `nomad alloc status` [[GH-6609](https://github.com/hashicorp/nomad/issues/6609)] * cli: Fixed a bug where `nomad alloc exec` fails if stdout is being redirected and not a TTY [[GH-6684](https://github.com/hashicorp/nomad/issues/6684)] * cli: Fixed a bug where a cli user may fail to query FS/Allocation API endpoints if they lack `node:read` capability [[GH-6423](https://github.com/hashicorp/nomad/issues/6423)] * client: client: Return empty values when host stats fail [[GH-6349](https://github.com/hashicorp/nomad/issues/6349)] * client: Fixed a bug where a client may not restart dead internal processes upon client's restart on Windows [[GH-6426](https://github.com/hashicorp/nomad/issues/6426)] * consul/connect: Fixed registering multiple Connect-enabled services in the same task group [[GH-6646](https://github.com/hashicorp/nomad/issues/6646)] * drivers: Fixed a bug where client may panic if a restored task failed to shutdown cleanly [[GH-6763](https://github.com/hashicorp/nomad/issues/6763)] * driver/exec: Fixed a bug where exec tasks can spawn processes that live beyond task lifecycle [[GH-6722](https://github.com/hashicorp/nomad/issues/6722)] * driver/docker: Added mechanism for detecting running unexpectedly running docker containers [[GH-6325](https://github.com/hashicorp/nomad/issues/6325)] * scheduler: Changes to devices in resource stanza should cause rescheduling [[GH-6644](https://github.com/hashicorp/nomad/issues/6644)] * scheduler: Fixed a bug that allowed inplace updates after affinity or spread were changed [[GH-6703](https://github.com/hashicorp/nomad/issues/6703)] * ui: Fixed client sorting [[GH-6817](https://github.com/hashicorp/nomad/issues/6817)] * vault: Allow overriding implicit Vault version constraint [[GH-6687](https://github.com/hashicorp/nomad/issues/6687)] * vault: Supported Vault auth role's new fields, `token_period` and `token_explicit_max_ttl` [[GH-6574](https://github.com/hashicorp/nomad/issues/6574)], [[GH-6580](https://github.com/hashicorp/nomad/issues/6580)] ## 0.10.1 (November 4, 2019) BUG FIXES: * core: Fixed server panic when upgrading from 0.8 -> 0.10 and performing an inplace update of an allocation. [[GH-6541](https://github.com/hashicorp/nomad/issues/6541)] * api: Fixed panic when submitting Connect-enabled job without using a bridge network [[GH-6575](https://github.com/hashicorp/nomad/issues/6575)] * client: Fixed client panic when upgrading from 0.8 -> 0.10 and performing an inplace update of an allocation. [[GH-6605](https://github.com/hashicorp/nomad/issues/6605)] ## 0.10.0 (October 22, 2019) FEATURES: * **Consul Connect**: Nomad may now register Consul Connect services and manages an Envoy proxy sidecar to provide secured service-to-service communication. * **Network Namespaces**: Task Groups may now define a shared network namespace. Each allocation will receive its own network namespace and loopback interface. Ports may be forwarded from the host into the network namespace. * **Host Volumes**: Nomad expanded support of stateful workloads through locally mounted storage volumes. * **UI Allocation File Explorer**: Nomad UI enhanced operability with a visual file system explorer for allocations. IMPROVEMENTS: * core: Added rolling deployments for service jobs by default and max_parallel=0 disables deployments [[GH-6191](https://github.com/hashicorp/nomad/pull/6100)] * agent: Allowed the job GC interval to be configured [[GH-5978](https://github.com/hashicorp/nomad/issues/5978)] * agent: Added `log_level` to be reloaded on SIGHUP [[GH-5996](https://github.com/hashicorp/nomad/pull/5996)] * api: Added follow parameter to file streaming endpoint to support older browsers [[GH-6049](https://github.com/hashicorp/nomad/issues/6049)] * client: Upgraded `go-getter` to support GCP links [[GH-6215](https://github.com/hashicorp/nomad/pull/6215)] * client: Remove consul service stanza from `job init --short` jobspec [[GH-6179](https://github.com/hashicorp/nomad/issues/6179)] * drivers: Exposed namespace as `NOMAD_NAMESPACE` environment variable in running tasks [[GH-6192](https://github.com/hashicorp/nomad/pull/6192)] * metrics: Added job status (pending, running, dead) metrics [[GH-6003](https://github.com/hashicorp/nomad/issues/6003)] * metrics: Added status and scheduling ability to client metrics [[GH-6130](https://github.com/hashicorp/nomad/issues/6130)] * server: Added an option to configure job GC interval [[GH-5978](https://github.com/hashicorp/nomad/issues/5978)] * ui: Added allocation filesystem explorer [[GH-5871](https://github.com/hashicorp/nomad/pull/5871)] * ui: Added creation time to evaluations table [[GH-6050](https://github.com/hashicorp/nomad/pull/6050)] BUG FIXES: * cli: Fixed `nomad run ...` on Windows so it works with unprivileged accounts [[GH-6009](https://github.com/hashicorp/nomad/issues/6009)] * client: Fixed a bug in client fingerprinting on 32-bit nodes [[GH-6239](https://github.com/hashicorp/nomad/issues/6239)] * client: Fixed a bug where completed allocations may re-run after client restart [[GH-6216](https://github.com/hashicorp/nomad/issues/6216)] * client: Fixed failure to start if another client is already running with the same data directory [[GH-6348](https://github.com/hashicorp/nomad/pull/6348)] * client: Fixed a panic that may occur when an `nomad alloc exec` is initiated while process is terminating [[GH-6065](https://github.com/hashicorp/nomad/issues/6065)] * devices: Fixed a bug causing CPU usage spike when a device is detected [[GH-6201](https://github.com/hashicorp/nomad/issues/6201)] * drivers: Allowd user-defined environment variable keys to contain dashes [[GH-6080](https://github.com/hashicorp/nomad/issues/6080)] * driver/docker: Set gc image_delay default to 3 minutes [[GH-6078](https://github.com/hashicorp/nomad/pull/6078)] * driver/docker: Improved docker driver handling of container creation or starting failures [[GH-6326](https://github.com/hashicorp/nomad/issues/6326)], [[GH-6346](https://github.com/hashicorp/nomad/issues/6346)] * ui: Fixed a bug where the allocation log viewer would render HTML or hide content that matched XML syntax [[GH-6048](https://github.com/hashicorp/nomad/issues/6048)] * ui: Fixed a bug where allocation log viewer doesn't show all content in Firefox [[GH-6466](https://github.com/hashicorp/nomad/issues/6466)] * ui: Fixed navigation via clicking recent allocation row [[GH-6087](https://github.com/hashicorp/nomad/pull/6087)] * ui: Fixed a bug where the allocation log viewer would render HTML or hide content that matched XML syntax [[GH-6048](https://github.com/hashicorp/nomad/issues/6048)] * ui: Fixed a bug where allocation log viewer doesn't show all content in Firefox [[GH-6466](https://github.com/hashicorp/nomad/issues/6466)] ## 0.9.7 (December 4, 2019) BUG FIXES: * core: Fixed server panic caused by a plan evicting and preempting allocs on a node [[GH-6792](https://github.com/hashicorp/nomad/issues/6792)] ## 0.9.6 (October 7, 2019) SECURITY: * core: Redacted replication token in agent/self API endpoint. The replication token is a management token that can be used for further privilege escalation. CVE-2019-12741 [[GH-6430](https://github.com/hashicorp/nomad/issues/6430)] * core: Fixed a bug where a user may start raw_exec task on clients despite driver being disabled. CVE-2019-15928 [[GH-6227](https://github.com/hashicorp/nomad/issues/6227)] [[GH-6431](https://github.com/hashicorp/nomad/issues/6431)] * enterprise/acl: Fix ACL access checks in Nomad Enterprise where users may query allocation information and perform lifecycle actions in namespaces they are not authorized to. CVE-2019-16742 [[GH-6432](https://github.com/hashicorp/nomad/issues/6432)] IMPROVEMENTS: * client: Reduced memory footprint of nomad logging and executor processes [[GH-6341](https://github.com/hashicorp/nomad/issues/6341)] BUG FIXES: * core: Fixed a bug where scheduler may schedule an allocation on a node without required drivers [[GH-6227](https://github.com/hashicorp/nomad/issues/6227)] * client: Fixed a bug where completed allocations may re-run after client restart [[GH-6216](https://github.com/hashicorp/nomad/issues/6216)] [[GH-6207](https://github.com/hashicorp/nomad/issues/6207)] * devices: Fixed a bug causing CPU usage spike when a device is detected [[GH-6201](https://github.com/hashicorp/nomad/issues/6201)] * drivers: Fixed port mapping for docker and qemu drivers [[GH-6251](https://github.com/hashicorp/nomad/pull/6251)] * drivers/docker: Fixed a case where a `nomad alloc exec` would never time out [[GH-6144](https://github.com/hashicorp/nomad/pull/6144)] * ui: Fixed a bug where allocation log viewer doesn't show all content. [[GH-6048](https://github.com/hashicorp/nomad/issues/6048)] ## 0.9.5 (21 August 2019) SECURITY: * client/template: Fix security vulnerabilities associated with task template rendering (CVE-2019-14802), introduced in Nomad 0.5.0 [[GH-6055](https://github.com/hashicorp/nomad/issues/6055)] [[GH-6075](https://github.com/hashicorp/nomad/issues/6075)] * client/artifact: Fix a privilege escalation in the `exec` driver exploitable by artifacts with setuid permissions (CVE-2019-14803) [[GH-6176](https://github.com/hashicorp/nomad/issues/6176)] __BACKWARDS INCOMPATIBILITIES:__ * client/template: When rendering a task template, only task environment variables are included by default. [[GH-6055](https://github.com/hashicorp/nomad/issues/6055)] * client/template: When rendering a task template, the `plugin` function is no longer permitted by default and will raise an error. [[GH-6075](https://github.com/hashicorp/nomad/issues/6075)] * client/template: When rendering a task template, path parameters for the `file` function will be restricted to the task directory by default. Relative paths or symlinks that point outside the task directory will raise an error. [[GH-6075](https://github.com/hashicorp/nomad/issues/6075)] IMPROVEMENTS: * core: Added create and modify timestamps to evaluations [[GH-5881](https://github.com/hashicorp/nomad/pull/5881)] BUG FIXES: * api: Fixed job region to default to client node region if none provided [[GH-6064](https://github.com/hashicorp/nomad/pull/6064)] * ui: Fixed links containing IPv6 addresses to include required square brackets [[GH-6007](https://github.com/hashicorp/nomad/pull/6007)] * vault: Fix deadlock when reloading server Vault configuration [[GH-6082](https://github.com/hashicorp/nomad/issues/6082)] ## 0.9.4 (July 30, 2019) IMPROVEMENTS: * api: Inferred content type of file in alloc filesystem stat endpoint [[GH-5907](https://github.com/hashicorp/nomad/issues/5907)] * api: Used region from job hcl when not provided as query parameter in job registration and plan endpoints [[GH-5664](https://github.com/hashicorp/nomad/pull/5664)] * core: Deregister nodes in batches rather than one at a time [[GH-5784](https://github.com/hashicorp/nomad/pull/5784)] * core: Removed deprecated upgrade path code pertaining to older versions of Nomad [[GH-5894](https://github.com/hashicorp/nomad/issues/5894)] * core: System jobs that fail because of resource availability are retried when resources are freed [[GH-5900](https://github.com/hashicorp/nomad/pull/5900)] * core: Support reloading log level in agent via SIGHUP [[GH-5996](https://github.com/hashicorp/nomad/issues/5996)] * client: Improved task event display message to include kill time out [[GH-5943](https://github.com/hashicorp/nomad/issues/5943)] * client: Removed extraneous information to improve formatting for hcl parsing error messages [[GH-5972](https://github.com/hashicorp/nomad/pull/5972)] * driver/docker: Added logging defaults to use json-file log driver with log rotation [[GH-5846](https://github.com/hashicorp/nomad/pull/5846)] * metrics: Added namespace label as appropriate to metrics [[GH-5847](https://github.com/hashicorp/nomad/issues/5847)] * ui: Added page titles [[GH-5924](https://github.com/hashicorp/nomad/pull/5924)] * ui: Added buttons to copy client and allocation UUIDs [[GH-5926](https://github.com/hashicorp/nomad/pull/5926)] * ui: Moved client status, draining, and eligibility fields into single state column [[GH-5789](https://github.com/hashicorp/nomad/pull/5789)] BUG FIXES: * core: Ensure plans are evaluated against a new enough snapshot index [[GH-5791](https://github.com/hashicorp/nomad/issues/5791)] * core: Handle error case when attempting to stop a non-existent allocation [[GH-5865](https://github.com/hashicorp/nomad/issues/5865)] * core: Improved job spec parsing error messages for variable interpolation failures [[GH-5844](https://github.com/hashicorp/nomad/issues/5844)] * core: Fixed a bug where nomad log and exec requests may time out or fail in tls enabled clusters [[GH-5954](https://github.com/hashicorp/nomad/issues/5954)]. * client: Fixed a bug where consul service health checks may flap on client restart [[GH-5837](https://github.com/hashicorp/nomad/issues/5837)] * client: Fixed a bug where too many check-based restarts would deadlock the client [[GH-5975](https://github.com/hashicorp/nomad/issues/5975)] * client: Fixed a bug where successfully completed tasks may restart on client restart [[GH-5890](https://github.com/hashicorp/nomad/issues/5890)] * client: Fixed a bug where stats of external driver plugins aren't collected on plugin restart [[GH-5948](https://github.com/hashicorp/nomad/issues/5948)] * client: Fixed an issue where an alloc remains in pending state if nomad fails to create alloc directory [[GH-5905](https://github.com/hashicorp/nomad/issues/5905)] * client: Fixed an issue where client may kill running allocs if the client and the leader are restarting simultaneously [[GH-5906](//github.com/hashicorp/nomad/issues/5906)] * client: Fixed regression that prevented registering multiple services with the same name but different ports in Consul correctly [[GH-5829](https://github.com/hashicorp/nomad/issues/5829)] * client: Fixed a race condition when performing local task restarts that would result in incorrect task not found errors on Windows [[GH-5899](https://github.com/hashicorp/nomad/pull/5889)] * client: Reduce CPU usage on clients running many tasks on Linux [[GH-5951](https://github.com/hashicorp/nomad/pull/5951)] * client: Updated consul-template dependency to address issue with anonymous requests [[GH-5976](https://github.com/hashicorp/nomad/issues/5976)] * driver: Fixed an issue preventing local task restarts on Windows [[GH-5864](https://github.com/hashicorp/nomad/pull/5864)] * driver: Fixed an issue preventing external driver plugins from launching executor process [[GH-5726](https://github.com/hashicorp/nomad/issues/5726)] * driver/docker: Fixed a bug mounting relative paths on Windows [[GH-5811](https://github.com/hashicorp/nomad/issues/5811)] * driver/exec: Upgraded libcontainer dependency to avoid zombie `runc:[1:CHILD]]` processes [[GH-5851](https://github.com/hashicorp/nomad/issues/5851)] * metrics: Added metrics for raft and state store indexes. [[GH-5841](https://github.com/hashicorp/nomad/issues/5841)] * metrics: Upgrade prometheus client to avoid label conflicts [[GH-5850](https://github.com/hashicorp/nomad/issues/5850)] * ui: Fixed ability to click sort arrow to change sort direction [[GH-5833](https://github.com/hashicorp/nomad/pull/5833)] ## 0.9.3 (June 12, 2019) BUG FIXES: * core: Fixed a panic that occurs if a job is updated with new task groups [[GH-5805](https://github.com/hashicorp/nomad/issues/5805)] * core: Update node's `StatusUpdatedAt` when node drain or eligibility changes [[GH-5746](https://github.com/hashicorp/nomad/issues/5746)] * core: Fixed a panic that may occur when preempting jobs for network resources [[GH-5794](https://github.com/hashicorp/nomad/issues/5794)] * core: Fixed a config parsing issue when client metadata contains a boolean value [[GH-5802](https://github.com/hashicorp/nomad/issues/5802)] * core: Fixed a config parsing issue where consul, vault, and autopilot stanzas break when using a config directory [[GH-5817](https://github.com/hashicorp/nomad/issues/5817)] * api: Allow sumitting alloc restart requests with an empty body [[GH-5823](https://github.com/hashicorp/nomad/pull/5823)] * client: Fixed an issue where task restart attempts is not honored properly [[GH-5737](https://github.com/hashicorp/nomad/issues/5737)] * client: Fixed a panic that occurs when a 0.9.2 client is running with 0.8 nomad servers [[GH-5812](https://github.com/hashicorp/nomad/issues/5812)] * client: Fixed an issue with cleaning up consul service registration entries when tasks fail to start. [[GH-5821](https://github.com/hashicorp/nomad/pull/5821)] ## 0.9.2 (June 5, 2019) SECURITY: * driver/exec: Fix privilege escalation issue introduced in Nomad 0.9.0. In Nomad 0.9.0 and 0.9.1, exec tasks by default run as `nobody` but with elevated capabilities, allowing tasks to perform privileged linux operations and potentially escalate permissions. (CVE-2019-12618) [[GH-5728](https://github.com/hashicorp/nomad/pull/5728)] __BACKWARDS INCOMPATIBILITIES:__ * api: The `api` package removed `Config.SetTimeout` and `Config.ConfigureTLS` functions, intended to be used internally only. [[GH-5275](https://github.com/hashicorp/nomad/pull/5275)] * api: The [job deployments](https://www.nomadproject.io/api/jobs.html#list-job-deployments) endpoint now filters out deployments associated with older instances of the job. This can happen if jobs are purged and recreated with the same id. To get all deployments irrespective of creation time, add `all=true`. The `nomad job deployment`CLI also defaults to doing this filtering. [[GH-5702](https://github.com/hashicorp/nomad/issues/5702)] * client: The format of service IDs in Consul has changed. If you rely upon Nomad's service IDs (*not* service names; those are stable), you will need to update your code. [[GH-5536](https://github.com/hashicorp/nomad/pull/5536)] * client: The format of check IDs in Consul has changed. If you rely upon Nomad's check IDs you will need to update your code. [[GH-5536](https://github.com/hashicorp/nomad/pull/5536)] * client: On startup a client will reattach to running tasks as before but will not restart exited tasks. Exited tasks will be restarted only after the client has reestablished communication with servers. System jobs will always be restarted. [[GH-5669](https://github.com/hashicorp/nomad/pull/5669)] FEATURES: * core: Add `nomad alloc stop` command to reschedule allocs [[GH-5512](https://github.com/hashicorp/nomad/pull/5512)] * core: Add `nomad alloc signal` command to signal allocs and tasks [[GH-5515](https://github.com/hashicorp/nomad/pull/5515)] * core: Add `nomad alloc restart` command to restart allocs and tasks [[GH-5502](https://github.com/hashicorp/nomad/pull/5502)] * code: Add `nomad alloc exec` command for debugging and running commands in a alloc [[GH-5632](https://github.com/hashicorp/nomad/pull/5632)] * core/enterprise: Preemption capabilities for batch and service jobs * ui: Preemption reporting everywhere where allocations are shown and as part of the plan step of job submit [[GH-5594](https://github.com/hashicorp/nomad/issues/5594)] * ui: Ability to search clients list by class, status, datacenter, or eligibility flags [[GH-5318](https://github.com/hashicorp/nomad/issues/5318)] * ui: Ability to search jobs list by type, status, datacenter, or prefix [[GH-5236](https://github.com/hashicorp/nomad/issues/5236)] * ui: Ability to stop and restart allocations [[GH-5734](https://github.com/hashicorp/nomad/issues/5734)] * ui: Ability to restart tasks [[GH-5734](https://github.com/hashicorp/nomad/issues/5734)] * vault: Add initial support for Vault namespaces [[GH-5520](https://github.com/hashicorp/nomad/pull/5520)] IMPROVEMENTS: * core: Add `-verbose` flag to `nomad status` wrapper command [[GH-5516](https://github.com/hashicorp/nomad/pull/5516)] * core: Add ability to filter job deployments by most recent version of job [[GH-5702](https://github.com/hashicorp/nomad/issues/5702)] * core: Add node name to output of `nomad node status` command in verbose mode [[GH-5224](https://github.com/hashicorp/nomad/pull/5224)] * core: Reduce the size of the raft transaction for plans by only sending fields updated by the plan applier [[GH-5602](https://github.com/hashicorp/nomad/pull/5602)] * core: Add job update `auto_promote` flag, which causes deployments to promote themselves when all canaries become healthy [[GH-5719](https://github.com/hashicorp/nomad/pull/5719)] * api: Support configuring `http.Client` used by golang `api` package [[GH-5275](https://github.com/hashicorp/nomad/pull/5275)] * api: Add preemption related fields to API results that return an allocation list. [[GH-5580](https://github.com/hashicorp/nomad/pull/5580)] * api: Add additional config options to scheduler configuration endpoint to disable preemption [[GH-5628](https://github.com/hashicorp/nomad/issues/5628)] * cli: Add acl token list command [[GH-5557](https://github.com/hashicorp/nomad/issues/5557)] * client: Reduce unnecessary lost nodes on server failure [[GH-5654](https://github.com/hashicorp/nomad/issues/5654)] * client: Canary Promotion no longer causes services registered in Consul to become unhealthy [[GH-4566](https://github.com/hashicorp/nomad/issues/4566)] * client: Allow use of maintenance mode and externally registered checks against Nomad-registered consul services [[GH-4537](https://github.com/hashicorp/nomad/issues/4537)] * driver/exec: Fixed an issue causing large memory consumption for light processes [[GH-5437](https://github.com/hashicorp/nomad/pull/5437)] * telemetry: Add `client.allocs.memory.allocated` metric to expose allocated task memory in bytes. [[GH-5492](https://github.com/hashicorp/nomad/issues/5492)] * ui: Colored log support [[GH-5620](https://github.com/hashicorp/nomad/issues/5620)] * ui: Upgraded from Ember 2.18 to 3.4 [[GH-5544](https://github.com/hashicorp/nomad/issues/5544)] * ui: Replace XHR cancellation by URL with XHR cancellation by token [[GH-5721](https://github.com/hashicorp/nomad/issues/5721)] BUG FIXES: * core: Fixed accounting of allocated resources in metrics. [[GH-5637](https://github.com/hashicorp/nomad/issues/5637)] * core: Fixed disaster recovering with raft 3 protocol peers.json [[GH-5629](https://github.com/hashicorp/nomad/issues/5629)], [[GH-5651](https://github.com/hashicorp/nomad/issues/5651)] * core: Fixed a panic that may occur when preempting service jobs [[GH-5545](https://github.com/hashicorp/nomad/issues/5545)] * core: Fixed an edge case that caused division by zero when computing spread score [[GH-5713](https://github.com/hashicorp/nomad/issues/5713)] * core: Change configuration parsing to use the HCL library's decode, improving JSON support [[GH-1290](https://github.com/hashicorp/nomad/issues/1290)] * core: Fix a case where non-leader servers would have an ever growing number of waiting evaluations [[GH-5699](https://github.com/hashicorp/nomad/pull/5699)] * cli: Fix output and exit status for system jobs with constraints [[GH-2381](https://github.com/hashicorp/nomad/issues/2381)] and [[GH-5169](https://github.com/hashicorp/nomad/issues/5169)] * client: Fix network fingerprinting to honor manual configuration [[GH-2619](https://github.com/hashicorp/nomad/issues/2619)] * client: Job validation now checks that the datacenter field does not contain empty strings [[GH-5665](https://github.com/hashicorp/nomad/pull/5665)] * client: Fix network port mapping related environment variables when running with Nomad 0.8 servers [[GH-5587](https://github.com/hashicorp/nomad/issues/5587)] * client: Fix issue with terminal state deployments being modified when allocation subsequently fails [[GH-5645](https://github.com/hashicorp/nomad/issues/5645)] * driver/docker: Fix regression around image GC [[GH-5768](https://github.com/hashicorp/nomad/issues/5768)] * metrics: Fixed stale metrics [[GH-5540](https://github.com/hashicorp/nomad/issues/5540)] * vault: Fix renewal time to be 1/2 lease duration with jitter [[GH-5479](https://github.com/hashicorp/nomad/issues/5479)] ## 0.9.1 (April 29, 2019) BUG FIXES: * core: Fix bug with incorrect metrics on pending allocations [[GH-5541](https://github.com/hashicorp/nomad/pull/5541)] * client: Fix issue with recovering from logmon failures [[GH-5577](https://github.com/hashicorp/nomad/pull/5577)], [[GH-5616](https://github.com/hashicorp/nomad/pull/5616)] * client: Fix deadlock on client startup after reboot [[GH-5568](https://github.com/hashicorp/nomad/pull/5568)] * client: Fix issue with node registration where newly registered nodes would not run system jobs [[GH-5585](https://github.com/hashicorp/nomad/pull/5585)] * driver/docker: Fix regression around volume handling [[GH-5572](https://github.com/hashicorp/nomad/pull/5572)] * driver/docker: Fix regression in which logs aren't collected for docker container with `tty` set to true [[GH-5609](https://github.com/hashicorp/nomad/pull/5609)] * driver/exec: Fix an issue where raw_exec and exec processes are leaked when nomad agent is restarted [[GH-5598](https://github.com/hashicorp/nomad/pull/5598)] ## 0.9.0 (April 9, 2019) __BACKWARDS INCOMPATIBILITIES:__ * core: Drop support for CentOS/RHEL 6. glibc >= 2.14 is required. * core: Switch to structured logging using [go-hclog](https://github.com/hashicorp/go-hclog). If you have tooling that parses Nomad's logs, the format of logs has changed and your tools may need updating. * core: IOPS as a resource is now deprecated [[GH-4970](https://github.com/hashicorp/nomad/issues/4970)]. Nomad continues to parse IOPS in jobs to allow job authors time to remove iops from their jobs. * core: Allow the != constraint to match against keys that do not exist [[GH-4875](https://github.com/hashicorp/nomad/pull/4875)] * client: Task config validation is more strict in 0.9. For example unknown parameters in stanzas under the task config were ignored in previous versions but in 0.9 this will cause a task failure. * client: Task config interpolation requires names to be valid identifiers (`node.region` or `NOMAD_DC`). Interpolating other variables requires a new indexing syntax: `env[".invalid.identifier."]`. [[GH-4843](https://github.com/hashicorp/nomad/issues/4843)] * client: Node metadata variables must have valid identifiers, whether specified in the config file (`client.meta` stanza) or on the command line (`-meta`). [[GH-5158](https://github.com/hashicorp/nomad/pull/5158)] * driver/lxc: The LXC driver is no longer packaged with Nomad and is instead distributed separately as a driver plugin. Further, the LXC driver codebase is now in a separate [repository](https://github.com/hashicorp/nomad-driver-lxc). If you are using LXC, please follow the 0.9.0 upgrade guide as you will have to install the LXC driver before conducting an in-place upgrade to Nomad 0.9.0 [[GH-5162](https://github.com/hashicorp/nomad/issues/5162)] FEATURES: * **Affinities and Spread**: Jobs may now specify affinities towards certain node attributes. Affinities act as soft constraints, and inform the scheduler that the job has a preference for certain node properties. The new spread stanza informs the scheduler that allocations should be spread across a specific property such as datacenter or availability zone. This is useful to increase failure tolerance of critical applications. * **System Job Preemption**: System jobs may now preempt lower priority allocations. The ability to place system jobs on all targeted nodes is critical since system jobs often run applications that provide services for all allocations on the node. * **Driver Plugins**: Nomad now supports task drivers as plugins. Driver plugins operate the same as built-in drivers and can be developed and distributed independently from Nomad. * **Device Plugins**: Nomad now supports scheduling and mounting devices via device plugins. Device plugins expose hardware devices such as GPUs to Nomad and instruct the client on how to make them available to tasks. Device plugins can expose the health of devices, the devices attributes, and device usage statistics. Device plugins can be developed and distributed independently from Nomad. * **Nvidia GPU Device Plugin**: Nomad builds-in a Nvidia GPU device plugin to add out-of-the-box support for scheduling Nvidia GPUs. * **Client Refactor**: Major focus has been put in this release to refactor the Nomad Client codebase. The goal of the refactor has been to make the codebase more modular to increase developer velocity and testability. * **Mobile UI Views:** The side-bar navigation, breadcrumbs, and various other page elements are now responsively resized and repositioned based on your browser size. * **Job Authoring from the UI:** It is now possible to plan and submit new jobs, edit existing jobs, stop and start jobs, and promote canaries all from the UI. * **Improved Stat Tracking in UI:** The client detail, allocation detail, and task detail pages now have line charts that plot CPU and Memory usage changes over time. * **Structured Logging**: Nomad now uses structured logging with the ability to output logs in a JSON format. IMPROVEMENTS: * core: Added advertise address to client node meta data [[GH-4390](https://github.com/hashicorp/nomad/issues/4390)] * core: Added support for specifying node affinities. Affinities allow job operators to specify weighted placement preferences according to different node attributes [[GH-4512](https://github.com/hashicorp/nomad/issues/4512)] * core: Added support for spreading allocations across a specific attribute. Operators can specify spread target percentages across failure domains such as datacenter or rack [[GH-4512](https://github.com/hashicorp/nomad/issues/4512)] * core: Added preemption support for system jobs. System jobs can now preempt other jobs of lower priority. See [preemption](https://www.nomadproject.io/docs/internals/scheduling/preemption.html) for more details. [[GH-4794](https://github.com/hashicorp/nomad/pull/4794)] * acls: Allow support for using globs in namespace definitions [[GH-4982](https://github.com/hashicorp/nomad/pull/4982)] * agent: Support JSON log output [[GH-5173](https://github.com/hashicorp/nomad/issues/5173)] * api: Reduced api package dependencies [[GH-5213](https://github.com/hashicorp/nomad/pull/5213)] * client: Extend timeout to 60 seconds for Windows CPU fingerprinting [[GH-4441](https://github.com/hashicorp/nomad/pull/4441)] * client: Refactor client to support plugins and improve state handling [[GH-4792](https://github.com/hashicorp/nomad/pull/4792)] * client: Updated consul-template library to pick up recent fixes and improvements[[GH-4885](https://github.com/hashicorp/nomad/pull/4885)] * client: When retrying a failed artifact, do not download any successfully downloaded artifacts again [[GH-5322](https://github.com/hashicorp/nomad/issues/5322)] * client: Added service metadata tag that enables the Consul UI to show a Nomad icon for services registered by Nomad [[GH-4889](https://github.com/hashicorp/nomad/issues/4889)] * cli: Added support for coloured output on Windows [[GH-5342](https://github.com/hashicorp/nomad/pull/5342)] * driver/docker: Rename Logging `type` to `driver` [[GH-5372](https://github.com/hashicorp/nomad/pull/5372)] * driver/docker: Support logs when using Docker for Mac [[GH-4758](https://github.com/hashicorp/nomad/issues/4758)] * driver/docker: Added support for specifying `storage_opt` in the Docker driver [[GH-4908](https://github.com/hashicorp/nomad/pull/4908)] * driver/docker: Added support for specifying `cpu_cfs_period` in the Docker driver [[GH-4462](https://github.com/hashicorp/nomad/pull/4462)] * driver/docker: Added support for setting bind and tmpfs mounts in the Docker driver [[GH-4924](https://github.com/hashicorp/nomad/pull/4924)] * driver/docker: Report container images with user friendly name rather than underlying image ID [[GH-4926](https://github.com/hashicorp/nomad/pull/4926)] * driver/docker: Add support for collecting stats on Windows [[GH-5355](https://github.com/hashicorp/nomad/pull/5355)] drivers/docker: Report docker driver as undetected before first connecting to the docker daemon [[GH-5362](https://github.com/hashicorp/nomad/pull/5362)] * drivers: Added total memory usage to task resource metrics [[GH-5190](https://github.com/hashicorp/nomad/pull/5190)] * server/rpc: Reduce logging when undergoing temporary network errors such as hitting file descriptor limits [[GH-4974](https://github.com/hashicorp/nomad/issues/4974)] * server/vault: Tweaked logs to better identify vault connection errors [[GH-5228](https://github.com/hashicorp/nomad/pull/5228)] * server/vault: Added Vault token expiry info in `nomad status` CLI, and some improvements to token refresh process [[GH-4817](https://github.com/hashicorp/nomad/pull/4817)] * telemetry: All client metrics include a new `node_class` tag [[GH-3882](https://github.com/hashicorp/nomad/issues/3882)] * telemetry: Added new tags with value of child job id and parent job id for parameterized and periodic jobs [[GH-4392](https://github.com/hashicorp/nomad/issues/4392)] * ui: Improved JSON editor [[GH-4541](https://github.com/hashicorp/nomad/issues/4541)] * ui: Mobile friendly views [[GH-4536](https://github.com/hashicorp/nomad/issues/4536)] * ui: Filled out the styleguide [[GH-4468](https://github.com/hashicorp/nomad/issues/4468)] * ui: Support switching regions [[GH-4572](https://github.com/hashicorp/nomad/issues/4572)] * ui: Canaries can now be promoted from the UI [[GH-4616](https://github.com/hashicorp/nomad/issues/4616)] * ui: Stopped jobs can be restarted from the UI [[GH-4615](https://github.com/hashicorp/nomad/issues/4615)] * ui: Support widescreen format in alloc logs view [[GH-5400](https://github.com/hashicorp/nomad/pull/5400)] * ui: Gracefully handle errors from the stats end points [[GH-4833](https://github.com/hashicorp/nomad/issues/4833)] * ui: Added links to Jobs and Clients from the error page template [[GH-4850](https://github.com/hashicorp/nomad/issues/4850)] * ui: Jobs can be authored, planned, submitted, and edited from the UI [[GH-4600](https://github.com/hashicorp/nomad/issues/4600)] * ui: Display recent allocations on job page and introduce allocation tab [[GH-4529](https://github.com/hashicorp/nomad/issues/4529)] * ui: Refactored breadcrumbs and adjusted the breadcrumb paths on each page [[GH-4458](https://github.com/hashicorp/nomad/issues/4458)] * ui: Switching namespaces in the UI will now always "reset" back to the jobs list page [[GH-4533](https://github.com/hashicorp/nomad/issues/4533)] * ui: CPU and Memory metrics are plotted over time during a session in line charts on node detail, allocation detail, and task detail pages [[GH-4661](https://github.com/hashicorp/nomad/issues/4661)], [[GH-4718](https://github.com/hashicorp/nomad/issues/4718)], [[GH-4727](https://github.com/hashicorp/nomad/issues/4727)] BUG FIXES: * core: Removed some GPL code inadvertently added for macOS support [[GH-5202](https://github.com/hashicorp/nomad/pull/5202)] * core: Fix an issue where artifact checksums containing interpolated variables failed validation [[GH-4810](https://github.com/hashicorp/nomad/pull/4819)] * core: Fix an issue where job summaries for parent dispatch/periodic jobs were not being computed correctly [[GH-5205](https://github.com/hashicorp/nomad/pull/5205)] * core: Fix an issue where a canary allocation with a deployment no longer in the state store caused a panic [[GH-5259](https://github.com/hashicorp/nomad/pull/5259) * client: Fix an issue reloading the client config [[GH-4730](https://github.com/hashicorp/nomad/issues/4730)] * client: Fix an issue where driver attributes are not updated in node API responses if they change after after startup [[GH-4984](https://github.com/hashicorp/nomad/pull/4984)] * driver/docker: Fix a path traversal issue where mounting paths outside alloc dir might be possible despite `docker.volumes.enabled` set to false [[GH-4983](https://github.com/hashicorp/nomad/pull/4983)] * driver/raw_exec: Fix an issue where tasks that used an interpolated command in driver configuration would not start [[GH-4813](https://github.com/hashicorp/nomad/pull/4813)] * drivers: Fix a bug where exec and java drivers get reported as detected and healthy when nomad is not running as root and without cgroup support * quota: Fixed a bug in Nomad enterprise where quota specifications were not being replicated to non authoritative regions correctly. * scheduler: When dequeueing evals ensure workers wait to the proper Raft index [[GH-5381](https://github.com/hashicorp/nomad/issues/5381)] * scheduler: Allow schedulers to handle evaluations that are created due to previous evaluation failures [[GH-4712](https://github.com/hashicorp/nomad/issues/4712)] * server/api: Fixed bug when trying to route to a down node [[GH-5261](https://github.com/hashicorp/nomad/pull/5261)] * server/vault: Fixed bug in Vault token renewal that could panic on a malformed Vault response [[GH-4904](https://github.com/hashicorp/nomad/issues/4904)], [[GH-4937](https://github.com/hashicorp/nomad/pull/4937)] * template: Fix parsing of environment templates when destination path is interpolated [[GH-5253](https://github.com/hashicorp/nomad/issues/5253)] * ui: Fixes for viewing objects that contain dots in their names [[GH-4994](https://github.com/hashicorp/nomad/issues/4994)] * ui: Correctly labeled certain classes of unknown errors as 404 errors [[GH-4841](https://github.com/hashicorp/nomad/issues/4841)] * ui: Fixed an issue where searching while viewing a paginated table could display no results [[GH-4822](https://github.com/hashicorp/nomad/issues/4822)] * ui: Fixed an issue where the task group breadcrumb didn't always include the namesapce query param [[GH-4801](https://github.com/hashicorp/nomad/issues/4801)] * ui: Added an empty state for the tasks list on the allocation detail page, for when an alloc has no tasks [[GH-4860](https://github.com/hashicorp/nomad/issues/4860)] * ui: Fixed an issue where dispatched jobs would get the wrong template type which could cause runtime errors [[GH-4852](https://github.com/hashicorp/nomad/issues/4852)] * ui: Fixed an issue where distribution bar corners weren't rounded when there was only one or two slices in the chart [[GH-4507](https://github.com/hashicorp/nomad/issues/4507)] ## 0.8.7 (January 14, 2019) IMPROVEMENTS: * core: Added `filter_default`, `prefix_filter` and `disable_dispatched_job_summary_metrics` client options to improve metric filtering [[GH-4878](https://github.com/hashicorp/nomad/issues/4878)] * driver/docker: Support `bind` mount type in order to allow Windows users to mount absolute paths [[GH-4958](https://github.com/hashicorp/nomad/issues/4958)] BUG FIXES: * core: Fixed panic when Vault secret response is nil [[GH-4904](https://github.com/hashicorp/nomad/pull/4904)] [[GH-4937](https://github.com/hashicorp/nomad/pull/4937)] * core: Fixed issue with negative counts in job summary [[GH-4949](https://github.com/hashicorp/nomad/issues/4949)] * core: Fixed issue with handling duplicated blocked evaluations [[GH-4867](https://github.com/hashicorp/nomad/pull/4867)] * core: Fixed bug where some successfully completed jobs get re-run after job garbage collection [[GH-4861](https://github.com/hashicorp/nomad/pull/4861)] * core: Fixed bug in reconciler where allocs already stopped were being unnecessarily updated [[GH-4764](https://github.com/hashicorp/nomad/issues/4764)] * core: Fixed bug that affects garbage collection of batch jobs that are purged and resubmitted with the same id [[GH-4839](https://github.com/hashicorp/nomad/pull/4839)] * core: Fixed an issue with garbage collection where terminal but still running allocations could be garbage collected server side [[GH-4965](https://github.com/hashicorp/nomad/issues/4965)] * deployments: Fix an issue where a deployment with multiple task groups could be marked as failed when the first progress deadline was hit regardless of if that group was done deploying [[GH-4842](https://github.com/hashicorp/nomad/issues/4842)] ## 0.8.6 (September 26, 2018) IMPROVEMENTS: * core: Increased scheduling performance when annotating existing allocations [[GH-4713](https://github.com/hashicorp/nomad/issues/4713)] * core: Unique TriggerBy for evaluations that are created to place queued allocations [[GH-4716](https://github.com/hashicorp/nomad/issues/4716)] BUG FIXES: * core: Fix a bug in Nomad Enterprise where non-voting servers could get bootstrapped as voting servers [[GH-4702](https://github.com/hashicorp/nomad/issues/4702)] * core: Fix an issue where an evaluation could fail if an allocation was being rescheduled and the node containing it was at capacity [[GH-4713](https://github.com/hashicorp/nomad/issues/4713)] * core: Fix an issue in which schedulers would reject evaluations created when prior scheduling for a job failed [[GH-4712](https://github.com/hashicorp/nomad/issues/4712)] * cli: Fix a bug where enabling custom upgrade versions for autopilot was not being honored [[GH-4723](https://github.com/hashicorp/nomad/issues/4723)] * deployments: Fix an issue where the deployment watcher could create a high volume of evaluations [[GH-4709](https://github.com/hashicorp/nomad/issues/4709)] * vault: Fix a regression in which Nomad was only compatible with Vault versions greater than 0.10.0 [[GH-4698](https://github.com/hashicorp/nomad/issues/4698)] ## 0.8.5 (September 13, 2018) IMPROVEMENTS: * core: Failed deployments no longer block migrations [[GH-4659](https://github.com/hashicorp/nomad/issues/4659)] * client: Added option to prevent Nomad from removing containers when the task exits [[GH-4535](https://github.com/hashicorp/nomad/issues/4535)] BUG FIXES: * core: Reset queued allocation summary to zero when job stopped [[GH-4414](https://github.com/hashicorp/nomad/issues/4414)] * core: Fix inverted logic bug where if `disable_update_check` was enabled, update checks would be performed [[GH-4570](https://github.com/hashicorp/nomad/issues/4570)] * core: Fix panic due to missing synchronization in delayed evaluations heap [[GH-4632](https://github.com/hashicorp/nomad/issues/4632)] * core: Fix treating valid PEM files as invalid [[GH-4613](https://github.com/hashicorp/nomad/issues/4613)] * core: Fix panic in nomad job history when invoked with a job version that doesn't exist [[GH-4577](https://github.com/hashicorp/nomad/issues/4577)] * core: Fix issue with not properly closing connection multiplexer when its context is cancelled [[GH-4573](https://github.com/hashicorp/nomad/issues/4573)] * core: Upgrade vendored Vault client library to fix API incompatibility issue [[GH-4658](https://github.com/hashicorp/nomad/issues/4658)] * driver/docker: Fix kill timeout not being respected when timeout is over five minutes [[GH-4599](https://github.com/hashicorp/nomad/issues/4599)] * scheduler: Fix nil pointer dereference [[GH-4474](https://github.com/hashicorp/nomad/issues/4474)] * scheduler: Fix panic when allocation's reschedule policy doesn't exist [[GH-4647](https://github.com/hashicorp/nomad/issues/4647)] * client: Fix migrating ephemeral disks when TLS is enabled [[GH-4648](https://github.com/hashicorp/nomad/issues/4648)] ## 0.8.4 (June 11, 2018) IMPROVEMENTS: * core: Updated serf library to improve how leave intents are handled [[GH-4278](https://github.com/hashicorp/nomad/issues/4278)] * core: Add more descriptive errors when parsing agent TLS certificates [[GH-4340](https://github.com/hashicorp/nomad/issues/4340)] * core: Added TLS configuration option to prefer server's ciphersuites over clients[[GH-4338](https://github.com/hashicorp/nomad/issues/4338)] * core: Add the option for operators to configure TLS versions and allowed cipher suites. Default is a subset of safe ciphers and TLS 1.2 [[GH-4269](https://github.com/hashicorp/nomad/pull/4269)] * core: Add a new [progress_deadline](https://www.nomadproject.io/docs/job-specification/update.html#progress_deadline) parameter to support rescheduling failed allocations during a deployment. This allows operators to specify a configurable deadline before which a deployment should see healthy allocations [[GH-4259](https://github.com/hashicorp/nomad/issues/4259)] * core: Add a new [job eval](https://www.nomadproject.io/docs/commands/job/eval.html) CLI and API for forcing an evaluation of a job, given the job ID. The new CLI also includes an option to force reschedule failed allocations [[GH-4274](https://github.com/hashicorp/nomad/issues/4274)] * core: Canary allocations are tagged in Consul to enable using service tags to isolate canary instances during deployments [[GH-4259](https://github.com/hashicorp/nomad/issues/4259)] * core: Emit Node events for drain and eligibility operations as well as for missed heartbeats [[GH-4284](https://github.com/hashicorp/nomad/issues/4284)], [[GH-4291](https://github.com/hashicorp/nomad/issues/4291)], [[GH-4292](https://github.com/hashicorp/nomad/issues/4292)] * agent: Support go-discover for auto-joining clusters based on cloud metadata [[GH-4277](https://github.com/hashicorp/nomad/issues/4277)] * cli: Add node drain monitoring with new `-monitor` flag on node drain command [[GH-4260](https://github.com/hashicorp/nomad/issues/4260)] * cli: Add node drain details to node status [[GH-4247](https://github.com/hashicorp/nomad/issues/4247)] * client: Avoid splitting log line across two files [[GH-4282](https://github.com/hashicorp/nomad/issues/4282)] * command: Add -short option to init command that emits a minimal jobspec [[GH-4239](https://github.com/hashicorp/nomad/issues/4239)] * discovery: Support Consul gRPC health checks. [[GH-4251](https://github.com/hashicorp/nomad/issues/4251)] * driver/docker: OOM kill metric [[GH-4185](https://github.com/hashicorp/nomad/issues/4185)] * driver/docker: Pull image with digest [[GH-4298](https://github.com/hashicorp/nomad/issues/4298)] * driver/docker: Support Docker pid limits [[GH-4341](https://github.com/hashicorp/nomad/issues/4341)] * driver/docker: Add progress monitoring and inactivity detection to docker image pulls [[GH-4192](https://github.com/hashicorp/nomad/issues/4192)] * driver/raw_exec: Use cgroups to manage process tree for precise cleanup of launched processes [[GH-4350](https://github.com/hashicorp/nomad/issues/4350)] * env: Default interpolation of optional meta fields of parameterized jobs to an empty string rather than the field key. [[GH-3720](https://github.com/hashicorp/nomad/issues/3720)] * ui: Show node drain, node eligibility, and node drain strategy information in the Client list and Client detail pages [[GH-4353](https://github.com/hashicorp/nomad/issues/4353)] * ui: Show reschedule-event information for allocations that were server-side rescheduled [[GH-4254](https://github.com/hashicorp/nomad/issues/4254)] * ui: Show the running deployment Progress Deadlines on the Job Detail Page [[GH-4388](https://github.com/hashicorp/nomad/issues/4388)] * ui: Show driver health status and node events on the Client Detail Page [[GH-4294](https://github.com/hashicorp/nomad/issues/4294)] * ui: Fuzzy and tokenized search on the Jobs List Page [[GH-4201](https://github.com/hashicorp/nomad/issues/4201)] * ui: The stop job button looks more dangerous [[GH-4339](https://github.com/hashicorp/nomad/issues/4339)] BUG FIXES: * core: Clean up leaked deployments on restoration [[GH-4329](https://github.com/hashicorp/nomad/issues/4329)] * core: Fix regression to allow for dynamic Vault configuration reload [[GH-4395](https://github.com/hashicorp/nomad/issues/4395)] * core: Fix bug where older failed allocations of jobs that have been updated to a newer version were not being garbage collected [[GH-4313](https://github.com/hashicorp/nomad/issues/4313)] * core: Fix bug when upgrading an existing server to Raft protocol 3 that caused servers to never change their ID in the Raft configuration. [[GH-4349](https://github.com/hashicorp/nomad/issues/4349)] * core: Fix bug with scheduler not creating a new deployment when job is purged and re-added [[GH-4377](https://github.com/hashicorp/nomad/issues/4377)] * api/client: Fix potentially out of order logs and streamed file contents [[GH-4234](https://github.com/hashicorp/nomad/issues/4234)] * discovery: Fix flapping services when Nomad Server and Client point to the same Consul agent [[GH-4365](https://github.com/hashicorp/nomad/issues/4365)] * driver/docker: Fix docker credential helper support [[GH-4266](https://github.com/hashicorp/nomad/issues/4266)] * driver/docker: Fix panic when docker client configuration options are invalid [[GH-4303](https://github.com/hashicorp/nomad/issues/4303)] * driver/exec: Disable exec on non-linux platforms [[GH-4366](https://github.com/hashicorp/nomad/issues/4366)] * rpc: Fix RPC tunneling when running both client/server on one machine [[GH-4317](https://github.com/hashicorp/nomad/issues/4317)] * ui: Track the method in XHR tracking to prevent errant ACL error dialogs when stopping a job [[GH-4319](https://github.com/hashicorp/nomad/issues/4319)] * ui: Make the tasks list on the Allocation Detail Page look and behave like other lists [[GH-4387](https://github.com/hashicorp/nomad/issues/4387)] [[GH-4393](https://github.com/hashicorp/nomad/issues/4393)] * ui: Use the Network IP, not the Node IP, for task addresses [[GH-4369](https://github.com/hashicorp/nomad/issues/4369)] * ui: Use Polling instead of Streaming for logs in Safari [[GH-4335](https://github.com/hashicorp/nomad/issues/4335)] * ui: Track PlaceCanaries in deployment metrics [[GH-4325](https://github.com/hashicorp/nomad/issues/4325)] ## 0.8.3 (April 27, 2018) BUG FIXES: * core: Fix panic proxying node connections when the server does not have a connection to the node [[GH-4231](https://github.com/hashicorp/nomad/issues/4231)] * core: Fix bug with not updating ModifyIndex of allocations after updates to the `NextAllocation` field [[GH-4250](https://github.com/hashicorp/nomad/issues/4250)] ## 0.8.2 (April 26, 2018) IMPROVEMENTS: * api: Add /v1/jobs/parse api endpoint for rendering HCL jobs files as JSON [[GH-2782](https://github.com/hashicorp/nomad/issues/2782)] * api: Include reschedule tracking events in end points that return a list of allocations [[GH-4240](https://github.com/hashicorp/nomad/issues/4240)] * cli: Improve help text when invalid arguments are given [[GH-4176](https://github.com/hashicorp/nomad/issues/4176)] * client: Create new process group on process startup. [[GH-3572](https://github.com/hashicorp/nomad/issues/3572)] * discovery: Periodically sync services and checks with Consul [[GH-4170](https://github.com/hashicorp/nomad/issues/4170)] * driver/rkt: Enable stats collection for rkt tasks [[GH-4188](https://github.com/hashicorp/nomad/pull/4188)] * ui: Stop job button added to job detail pages [[GH-4189](https://github.com/hashicorp/nomad/pull/4189)] BUG FIXES: * core: Handle invalid cron specifications more gracefully [[GH-4224](https://github.com/hashicorp/nomad/issues/4224)] * core: Sort signals in implicit constraint avoiding unnecessary updates [[GH-4216](https://github.com/hashicorp/nomad/issues/4216)] * core: Improve tracking of node connections even if the address being used to contact the server changes [[GH-4222](https://github.com/hashicorp/nomad/issues/4222)] * core: Fix panic when doing a node drain effecting a job that has an allocation that was on a node that no longer exists [[GH-4215](https://github.com/hashicorp/nomad/issues/4215)] * api: Fix an issue in which the autopilot configuration could not be updated [[GH-4220](https://github.com/hashicorp/nomad/issues/4220)] * client: Populate access time and modify time when unarchiving tar archives that do not specify them explicitly [[GH-4217](https://github.com/hashicorp/nomad/issues/4217)] * driver/exec: Create process group for Windows process and send Ctrl-Break signal on Shutdown [[GH-4153](https://github.com/hashicorp/nomad/pull/4153)] * ui: Alloc stats will continue to poll after a request errors or returns an invalid response [[GH-4195](https://github.com/hashicorp/nomad/pull/4195)] ## 0.8.1 (April 17, 2018) BUG FIXES: * client: Fix a race condition while concurrently fingerprinting and accessing the node that could cause a panic [[GH-4166](https://github.com/hashicorp/nomad/issues/4166)] ## 0.8.0 (April 12, 2018) __BACKWARDS INCOMPATIBILITIES:__ * cli: node drain now blocks until the drain completes and all allocations on the draining node have stopped. Use -detach for the old behavior. * client: Periods (`.`) are no longer replaced with underscores (`_`) in environment variables as many applications rely on periods in environment variable names. [[GH-3760](https://github.com/hashicorp/nomad/issues/3760)] * client/metrics: The key emitted for tracking a client's uptime has changed from "uptime" to "client.uptime". Users monitoring this metric will have to switch to the new key name [[GH-4128](https://github.com/hashicorp/nomad/issues/4128)] * discovery: Prevent absolute URLs in check paths. The documentation indicated that absolute URLs are not allowed, but it was not enforced. Absolute URLs in HTTP check paths will now fail to validate. [[GH-3685](https://github.com/hashicorp/nomad/issues/3685)] * drain: Draining a node no longer stops all allocations immediately: a new [migrate stanza](https://www.nomadproject.io/docs/job-specification/migrate.html) allows jobs to specify how quickly task groups can be drained. A `-force` option can be used to emulate the old drain behavior. * jobspec: The default values for restart policy have changed. Restart policy mode defaults to "fail" and the attempts/time interval values have been changed to enable faster server side rescheduling. See [restart stanza](https://www.nomadproject.io/docs/job-specification/restart.html) for more information. * jobspec: Removed compatibility code that migrated pre Nomad 0.6.0 Update stanza syntax. All job spec files should be using update stanza fields introduced in 0.7.0 [[GH-3979](https://github.com/hashicorp/nomad/pull/3979/files)] IMPROVEMENTS: * core: Servers can now service client HTTP endpoints [[GH-3892](https://github.com/hashicorp/nomad/issues/3892)] * core: More efficient garbage collection of large batches of jobs [[GH-3982](https://github.com/hashicorp/nomad/issues/3982)] * core: Allow upgrading/downgrading TLS via SIGHUP on both servers and clients [[GH-3492](https://github.com/hashicorp/nomad/issues/3492)] * core: Node events are emitted for events such as node registration and heartbeating [[GH-3945](https://github.com/hashicorp/nomad/issues/3945)] * core: A set of features (Autopilot) has been added to allow for automatic operator-friendly management of Nomad servers. For more information about Autopilot, see the [Autopilot Guide](https://www.nomadproject.io/guides/cluster/autopilot.html). [[GH-3670](https://github.com/hashicorp/nomad/pull/3670)] * core: Failed tasks are automatically rescheduled according to user specified criteria. For more information on configuration, see the [Reshedule Stanza](https://www.nomadproject.io/docs/job-specification/reschedule.html) [[GH-3981](https://github.com/hashicorp/nomad/issues/3981)] * core: Servers can now service client HTTP endpoints [[GH-3892](https://github.com/hashicorp/nomad/issues/3892)] * core: Servers can now retry connecting to Vault to verify tokens without requiring a SIGHUP to do so [[GH-3957](https://github.com/hashicorp/nomad/issues/3957)] * core: Updated yamux library to pick up memory and CPU performance improvements [[GH-3980](https://github.com/hashicorp/nomad/issues/3980)] * core: Client stanza now supports overriding total memory [[GH-4052](https://github.com/hashicorp/nomad/issues/4052)] * core: Node draining is now able to migrate allocations in a controlled manner with parameters specified by the drain command and in job files using the migrate stanza [[GH-4010](https://github.com/hashicorp/nomad/issues/4010)] * acl: Increase token name limit from 64 characters to 256 [[GH-3888](https://github.com/hashicorp/nomad/issues/3888)] * cli: Node status and filesystem related commands do not require direct network access to the Nomad client nodes [[GH-3892](https://github.com/hashicorp/nomad/issues/3892)] * cli: Common commands highlighed [[GH-4027](https://github.com/hashicorp/nomad/issues/4027)] * cli: Colored error and warning outputs [[GH-4027](https://github.com/hashicorp/nomad/issues/4027)] * cli: All commands are grouped by subsystem [[GH-4027](https://github.com/hashicorp/nomad/issues/4027)] * cli: Use ISO_8601 time format for cli output [[GH-3814](https://github.com/hashicorp/nomad/pull/3814)] * cli: Clearer task event descriptions in `nomad alloc-status` when there are server side failures authenticating to Vault [[GH-3968](https://github.com/hashicorp/nomad/issues/3968)] * client: Allow '.' in environment variable names [[GH-3760](https://github.com/hashicorp/nomad/issues/3760)] * client: Improved handling of failed RPCs and heartbeat retry logic [[GH-4106](https://github.com/hashicorp/nomad/issues/4106)] * client: Refactor client fingerprint methods to a request/response format [[GH-3781](https://github.com/hashicorp/nomad/issues/3781)] * client: Enable periodic health checks for drivers. Initial support only includes the Docker driver. [[GH-3856](https://github.com/hashicorp/nomad/issues/3856)] * discovery: Allow `check_restart` to be specified in the `service` stanza [[GH-3718](https://github.com/hashicorp/nomad/issues/3718)] * discovery: Allow configuring names of Nomad client and server health checks [[GH-4003](https://github.com/hashicorp/nomad/issues/4003)] * discovery: Only log if Consul does not support TLSSkipVerify instead of dropping checks which relied on it. Consul has had this feature since 0.7.2 [[GH-3983](https://github.com/hashicorp/nomad/issues/3983)] * driver/docker: Support hard CPU limits [[GH-3825](https://github.com/hashicorp/nomad/issues/3825)] * driver/docker: Support advertising IPv6 addresses [[GH-3790](https://github.com/hashicorp/nomad/issues/3790)] * driver/docker; Support overriding image entrypoint [[GH-3788](https://github.com/hashicorp/nomad/issues/3788)] * driver/docker: Support adding or dropping capabilities [[GH-3754](https://github.com/hashicorp/nomad/issues/3754)] * driver/docker: Support mounting root filesystem as read-only [[GH-3802](https://github.com/hashicorp/nomad/issues/3802)] * driver/docker: Retry on Portworx "volume is attached on another node" errors [[GH-3993](https://github.com/hashicorp/nomad/issues/3993)] * driver/lxc: Add volumes config to LXC driver [[GH-3687](https://github.com/hashicorp/nomad/issues/3687)] * driver/rkt: Allow overriding group [[GH-3990](https://github.com/hashicorp/nomad/issues/3990)] * telemetry: Support DataDog tags [[GH-3839](https://github.com/hashicorp/nomad/issues/3839)] * ui: Specialized job detail pages for each job type (system, service, batch, periodic, parameterized, periodic instance, parameterized instance) [[GH-3829](https://github.com/hashicorp/nomad/issues/3829)] * ui: Allocation stats requests are made through the server instead of directly through clients [[GH-3908](https://github.com/hashicorp/nomad/issues/3908)] * ui: Allocation log requests fallback to using the server when the client can't be reached [[GH-3908](https://github.com/hashicorp/nomad/issues/3908)] * ui: All views poll for changes using long-polling via blocking queries [[GH-3936](https://github.com/hashicorp/nomad/issues/3936)] * ui: Dispatch payload on the parameterized instance job detail page [[GH-3829](https://github.com/hashicorp/nomad/issues/3829)] * ui: Periodic force launch button on the periodic job detail page [[GH-3829](https://github.com/hashicorp/nomad/issues/3829)] * ui: Allocation breadcrumbs now extend job breadcrumbs [[GH-3829](https://github.com/hashicorp/nomad/issues/3974)] * vault: Allow Nomad to create orphaned tokens for allocations [[GH-3992](https://github.com/hashicorp/nomad/issues/3992)] BUG FIXES: * core: Fix search endpoint forwarding for multi-region clusters [[GH-3680](https://github.com/hashicorp/nomad/issues/3680)] * core: Fix an issue in which batch jobs with queued placements and lost allocations could result in improper placement counts [[GH-3717](https://github.com/hashicorp/nomad/issues/3717)] * core: Fix an issue where an entire region leaving caused `nomad server-members` to fail with a 500 response [[GH-1515](https://github.com/hashicorp/nomad/issues/1515)] * core: Fix an issue in which multiple servers could be acting as a leader. A prominent side-effect being nodes TTLing incorrectly [[GH-3890](https://github.com/hashicorp/nomad/issues/3890)] * core: Fix an issue where jobs with the same name in a different namespace were not being blocked correctly [[GH-3972](https://github.com/hashicorp/nomad/issues/3972)] * cli: server member command handles failure to retrieve leader in remote regions [[GH-4087](https://github.com/hashicorp/nomad/issues/4087)] * client: Support IP detection of wireless interfaces on Windows [[GH-4011](https://github.com/hashicorp/nomad/issues/4011)] * client: Migrated ephemeral_disk's maintain directory permissions [[GH-3723](https://github.com/hashicorp/nomad/issues/3723)] * client: Always advertise driver IP when in driver address mode [[GH-3682](https://github.com/hashicorp/nomad/issues/3682)] * client: Preserve permissions on directories when expanding tarred artifacts [[GH-4129](https://github.com/hashicorp/nomad/issues/4129)] * client: Improve auto-detection of network interface when interface name has a space in it on Windows [[GH-3855](https://github.com/hashicorp/nomad/issues/3855)] * client/vault: Recognize renewing non-renewable Vault lease as fatal [[GH-3727](https://github.com/hashicorp/nomad/issues/3727)] * client/vault: Improved error handling of network errors with Vault [[GH-4100](https://github.com/hashicorp/nomad/issues/4100)] * config: Revert minimum CPU limit back to 20 from 100 [[GH-3706](https://github.com/hashicorp/nomad/issues/3706)] * config: Always add core scheduler to enabled schedulers and add invalid EnabledScheduler detection [[GH-3978](https://github.com/hashicorp/nomad/issues/3978)] * driver/exec: Properly disable swapping [[GH-3958](https://github.com/hashicorp/nomad/issues/3958)] * driver/lxc: Cleanup LXC containers after errors on container startup. [[GH-3773](https://github.com/hashicorp/nomad/issues/3773)] * ui: Always show the task name in the task recent events table on the allocation detail page. [[GH-3985](https://github.com/hashicorp/nomad/pull/3985)] * ui: Only show the placement failures section when there is a blocked evaluation. [[GH-3956](https://github.com/hashicorp/nomad/pull/3956)] * ui: Fix requests using client-side certificates in Firefox. [[GH-3728](https://github.com/hashicorp/nomad/pull/3728)] * ui: Fix ui on non-leaders when ACLs are enabled [[GH-3722](https://github.com/hashicorp/nomad/issues/3722)] ## 0.7.1 (December 19, 2017) __BACKWARDS INCOMPATIBILITIES:__ * client: The format of service IDs in Consul has changed. If you rely upon Nomad's service IDs (*not* service names; those are stable), you will need to update your code. [[GH-3632](https://github.com/hashicorp/nomad/issues/3632)] * config: Nomad no longer parses Atlas configuration stanzas. Atlas has been deprecated since earlier this year. If you have an Atlas stanza in your config file it will have to be removed. * config: Default minimum CPU configuration has been changed to 100 from 20. Jobs using the old minimum value of 20 will have to be updated. * telemetry: Hostname is now emitted via a tag rather than within the key name. To maintain old behavior during an upgrade path specify `backwards_compatible_metrics` in the telemetry configuration. IMPROVEMENTS: * core: Allow operators to reload TLS certificate and key files via SIGHUP [[GH-3479](https://github.com/hashicorp/nomad/issues/3479)] * core: Allow configurable stop signals for a task, when drivers support sending stop signals [[GH-1755](https://github.com/hashicorp/nomad/issues/1755)] * core: Allow agents to be run in `rpc_upgrade_mode` when migrating a cluster to TLS rather than changing `heartbeat_grace` * api: Allocations now track and return modify time in addition to create time [[GH-3446](https://github.com/hashicorp/nomad/issues/3446)] * api: Introduced new fields to track details and display message for task events, and deprecated redundant existing fields [[GH-3399](https://github.com/hashicorp/nomad/issues/3399)] * api: Environment variables are ignored during service name validation [[GH-3532](https://github.com/hashicorp/nomad/issues/3532)] * cli: Allocation create and modify times are displayed in a human readable relative format like `6 h ago` [[GH-3449](https://github.com/hashicorp/nomad/issues/3449)] * client: Support `address_mode` on checks [[GH-3619](https://github.com/hashicorp/nomad/issues/3619)] * client: Sticky volume migrations are now atomic. [[GH-3563](https://github.com/hashicorp/nomad/issues/3563)] * client: Added metrics to track state transitions of allocations [[GH-3061](https://github.com/hashicorp/nomad/issues/3061)] * client: When `network_interface` is unspecified use interface attached to default route [[GH-3546](https://github.com/hashicorp/nomad/issues/3546)] * client: Support numeric ports on services and checks when `address_mode="driver"` [[GH-3619](https://github.com/hashicorp/nomad/issues/3619)] * driver/docker: Detect OOM kill event [[GH-3459](https://github.com/hashicorp/nomad/issues/3459)] * driver/docker: Adds support for adding host device to container via `--device` [[GH-2938](https://github.com/hashicorp/nomad/issues/2938)] * driver/docker: Adds support for `ulimit` and `sysctl` options [[GH-3568](https://github.com/hashicorp/nomad/issues/3568)] * driver/docker: Adds support for StopTimeout (set to the same value as kill_timeout [[GH-3601](https://github.com/hashicorp/nomad/issues/3601)] * driver/rkt: Add support for passing through user [[GH-3612](https://github.com/hashicorp/nomad/issues/3612)] * driver/qemu: Support graceful shutdowns on unix platforms [[GH-3411](https://github.com/hashicorp/nomad/issues/3411)] * template: Updated to consul template 0.19.4 [[GH-3543](https://github.com/hashicorp/nomad/issues/3543)] * core/enterprise: Return 501 status code in Nomad Pro for Premium end points * ui: Added log streaming for tasks [[GH-3564](https://github.com/hashicorp/nomad/issues/3564)] * ui: Show the modify time for allocations [[GH-3607](https://github.com/hashicorp/nomad/issues/3607)] * ui: Added a dedicated Task page under allocations [[GH-3472](https://github.com/hashicorp/nomad/issues/3472)] * ui: Added placement failures to the Job Detail page [[GH-3603](https://github.com/hashicorp/nomad/issues/3603)] * ui: Warn uncaught exceptions to the developer console [[GH-3623](https://github.com/hashicorp/nomad/issues/3623)] BUG FIXES: * core: Fix issue in which restoring periodic jobs could fail when a leader election occurs [[GH-3646](https://github.com/hashicorp/nomad/issues/3646)] * core: Fix race condition in which rapid reprocessing of a blocked evaluation may lead to the scheduler not seeing the results of the previous scheduling event [[GH-3669](https://github.com/hashicorp/nomad/issues/3669)] * core: Fixed an issue where the leader server could get into a state where it was no longer performing the periodic leader loop duties after a barrier timeout error [[GH-3402](https://github.com/hashicorp/nomad/issues/3402)] * core: Fixes an issue with jobs that have `auto_revert` set to true, where reverting to a previously stable job that fails to start up causes an infinite cycle of reverts [[GH-3496](https://github.com/hashicorp/nomad/issues/3496)] * api: Apply correct memory default when task's do not specify memory explicitly [[GH-3520](https://github.com/hashicorp/nomad/issues/3520)] * cli: Fix passing Consul address via flags [[GH-3504](https://github.com/hashicorp/nomad/issues/3504)] * cli: Fix panic when running `keyring` commands [[GH-3509](https://github.com/hashicorp/nomad/issues/3509)] * client: Fix advertising services with tags that require URL escaping [[GH-3632](https://github.com/hashicorp/nomad/issues/3632)] * client: Fix a panic when restoring an allocation with a dead leader task [[GH-3502](https://github.com/hashicorp/nomad/issues/3502)] * client: Fix crash when following logs from a Windows node [[GH-3608](https://github.com/hashicorp/nomad/issues/3608)] * client: Fix service/check updating when just interpolated variables change [[GH-3619](https://github.com/hashicorp/nomad/issues/3619)] * client: Fix allocation accounting in GC and trigger GCs on allocation updates [[GH-3445](https://github.com/hashicorp/nomad/issues/3445)] * driver/docker: Fix container name conflict handling [[GH-3551](https://github.com/hashicorp/nomad/issues/3551)] * driver/rkt: Remove pods on shutdown [[GH-3562](https://github.com/hashicorp/nomad/issues/3562)] * driver/rkt: Don't require port maps when using host networking [[GH-3615](https://github.com/hashicorp/nomad/issues/3615)] * template: Fix issue where multiple environment variable templates would be parsed incorrectly when contents of one have changed after the initial rendering [[GH-3529](https://github.com/hashicorp/nomad/issues/3529)] * sentinel: (Nomad Enterprise) Fix an issue that could cause an import error when multiple Sentinel policies are applied * telemetry: Do not emit metrics for non-running tasks [[GH-3559](https://github.com/hashicorp/nomad/issues/3559)] * telemetry: Emit hostname as a tag rather than within the key name [[GH-3616](https://github.com/hashicorp/nomad/issues/3616)] * ui: Remove timezone text from timestamps [[GH-3621](https://github.com/hashicorp/nomad/issues/3621)] * ui: Allow cross-origin requests from the UI [[GH-3530](https://github.com/hashicorp/nomad/issues/3530)] * ui: Consistently use Clients instead of Nodes in copy [[GH-3466](https://github.com/hashicorp/nomad/issues/3466)] * ui: Fully expand the job definition on the Job Definition page [[GH-3631](https://github.com/hashicorp/nomad/issues/3631)] ## 0.7.0 (November 1, 2017) __BACKWARDS INCOMPATIBILITIES:__ * driver/rkt: Nomad now requires at least rkt version `1.27.0` for the rkt driver to function. Please update your version of rkt to at least this version. IMPROVEMENTS: * core: Capability based ACL system with authoritative region, providing federated ACLs. * core/enterprise: Sentinel integration for fine grain policy enforcement. * core/enterprise: Namespace support allowing jobs and their associated objects to be isolated from each other and other users of the cluster. * api: Allow force deregistration of a node [[GH-3447](https://github.com/hashicorp/nomad/issues/3447)] * api: New `/v1/agent/health` endpoint for health checks. * api: Metrics endpoint exposes Prometheus formatted metrics [[GH-3171](https://github.com/hashicorp/nomad/issues/3171)] * cli: Consul config option flags for nomad agent command [[GH-3327](https://github.com/hashicorp/nomad/issues/3327)] * discovery: Allow restarting unhealthy tasks with `check_restart` [[GH-3105](https://github.com/hashicorp/nomad/issues/3105)] * driver/rkt: Enable rkt driver to use address_mode = 'driver' [[GH-3256](https://github.com/hashicorp/nomad/issues/3256)] * telemetry: Add support for tagged metrics for Nomad clients [[GH-3147](https://github.com/hashicorp/nomad/issues/3147)] * telemetry: Add basic Prometheus configuration for a Nomad cluster [[GH-3186](https://github.com/hashicorp/nomad/issues/3186)] BUG FIXES: * core: Fix restoration of stopped periodic jobs [[GH-3201](https://github.com/hashicorp/nomad/issues/3201)] * core: Run deployment garbage collector on an interval [[GH-3267](https://github.com/hashicorp/nomad/issues/3267)] * core: Fix parameterized jobs occasionally showing status dead incorrectly [[GH-3460](https://github.com/hashicorp/nomad/issues/3460)] * core: Fix issue in which job versions above a threshold potentially wouldn't be stored [[GH-3372](https://github.com/hashicorp/nomad/issues/3372)] * core: Fix issue where node-drain with complete batch allocation would create replacement [[GH-3217](https://github.com/hashicorp/nomad/issues/3217)] * core: Allow batch jobs that have been purged to be rerun without a job specification change [[GH-3375](https://github.com/hashicorp/nomad/issues/3375)] * core: Fix issue in which batch allocations from previous job versions may not have been stopped properly. [[GH-3217](https://github.com/hashicorp/nomad/issues/3217)] * core: Fix issue in which allocations with the same name during a scale down/stop event wouldn't be properly stopped [[GH-3217](https://github.com/hashicorp/nomad/issues/3217)] * core: Fix a race condition in which scheduling results from one invocation of the scheduler wouldn't be considered by the next for the same job [[GH-3206](https://github.com/hashicorp/nomad/issues/3206)] * api: Sort /v1/agent/servers output so that output of Consul checks does not change [[GH-3214](https://github.com/hashicorp/nomad/issues/3214)] * api: Fix search handling of jobs with more than four hyphens and case were length could cause lookup error [[GH-3203](https://github.com/hashicorp/nomad/issues/3203)] * client: Improve the speed at which clients detect garbage collection events [[GH-3452](https://github.com/hashicorp/nomad/issues/3452)] * client: Fix lock contention that could cause a node to miss a heartbeat and be marked as down [[GH-3195](https://github.com/hashicorp/nomad/issues/3195)] * client: Fix data race that could lead to concurrent map read/writes during heartbeating and fingerprinting [[GH-3461](https://github.com/hashicorp/nomad/issues/3461)] * driver/docker: Fix docker user specified syslogging [[GH-3184](https://github.com/hashicorp/nomad/issues/3184)] * driver/docker: Fix issue where CPU usage statistics were artificially high [[GH-3229](https://github.com/hashicorp/nomad/issues/3229)] * client/template: Fix issue in which secrets would be renewed too aggressively [[GH-3360](https://github.com/hashicorp/nomad/issues/3360)] ## 0.6.3 (September 11, 2017) BUG FIXES: * api: Search handles prefix longer than allowed UUIDs [[GH-3138](https://github.com/hashicorp/nomad/issues/3138)] * api: Search endpoint handles even UUID prefixes with hyphens [[GH-3120](https://github.com/hashicorp/nomad/issues/3120)] * api: Don't merge empty update stanza from job into task groups [[GH-3139](https://github.com/hashicorp/nomad/issues/3139)] * cli: Sort task groups when displaying a deployment [[GH-3137](https://github.com/hashicorp/nomad/issues/3137)] * cli: Handle reading files that are in a symlinked directory [[GH-3164](https://github.com/hashicorp/nomad/issues/3164)] * cli: All status commands handle even UUID prefixes with hyphens [[GH-3122](https://github.com/hashicorp/nomad/issues/3122)] * cli: Fix autocompletion of paths that include directories on zsh [[GH-3129](https://github.com/hashicorp/nomad/issues/3129)] * cli: Fix job deployment -latest handling of jobs without deployments [[GH-3166](https://github.com/hashicorp/nomad/issues/3166)] * cli: Hide CLI commands not expected to be run by user from autocomplete suggestions [[GH-3177](https://github.com/hashicorp/nomad/issues/3177)] * cli: Status command honors exact job match even when it is the prefix of another job [[GH-3120](https://github.com/hashicorp/nomad/issues/3120)] * cli: Fix setting of TLSServerName for node API Client. This fixes an issue of contacting nodes that are using TLS [[GH-3127](https://github.com/hashicorp/nomad/issues/3127)] * client/template: Fix issue in which the template block could cause high load on Vault when secret lease duration was less than the Vault grace [[GH-3153](https://github.com/hashicorp/nomad/issues/3153)] * driver/docker: Always purge stopped containers [[GH-3148](https://github.com/hashicorp/nomad/issues/3148)] * driver/docker: Fix MemorySwappiness on Windows [[GH-3187](https://github.com/hashicorp/nomad/issues/3187)] * driver/docker: Fix issue in which mounts could parse incorrectly [[GH-3163](https://github.com/hashicorp/nomad/issues/3163)] * driver/docker: Fix issue where potentially incorrect syslog server address is used [[GH-3135](https://github.com/hashicorp/nomad/issues/3135)] * driver/docker: Fix server url passed to credential helpers and properly capture error output [[GH-3165](https://github.com/hashicorp/nomad/issues/3165)] * jobspec: Allow distinct_host constraint to have L/RTarget set [[GH-3136](https://github.com/hashicorp/nomad/issues/3136)] ## 0.6.2 (August 28, 2017) BUG FIXES: * api/cli: Fix logs and fs api and command [[GH-3116](https://github.com/hashicorp/nomad/issues/3116)] ## 0.6.1 (August 28, 2017) __BACKWARDS INCOMPATIBILITIES:__ * deployment: Specifying an update stanza with a max_parallel of zero is now a validation error. Please update the stanza to be greater than zero or remove the stanza as a zero parallelism update is not valid. IMPROVEMENTS: * core: Lost allocations replaced even if part of failed deployment [[GH-2961](https://github.com/hashicorp/nomad/issues/2961)] * core: Add autocomplete functionality for resources: allocations, evaluations, jobs, deployments and nodes [[GH-2964](https://github.com/hashicorp/nomad/issues/2964)] * core: `distinct_property` constraint can set the number of allocations that are allowed to share a property value [[GH-2942](https://github.com/hashicorp/nomad/issues/2942)] * core: Placing allocation counts towards placement limit fixing issue where rolling update could remove an unnecessary amount of allocations [[GH-3070](https://github.com/hashicorp/nomad/issues/3070)] * api: Redact Vault.Token from AgentSelf response [[GH-2988](https://github.com/hashicorp/nomad/issues/2988)] * cli: node-status displays node version [[GH-3002](https://github.com/hashicorp/nomad/issues/3002)] * cli: Disable color output when STDOUT is not a TTY [[GH-3057](https://github.com/hashicorp/nomad/issues/3057)] * cli: Add autocomplete functionality for flags for all CLI command [GH 3087] * cli: Add status command which takes any identifier and routes to the appropriate status command. * client: Unmount task directories when alloc is terminal [[GH-3006](https://github.com/hashicorp/nomad/issues/3006)] * client/template: Allow template to set Vault grace [[GH-2947](https://github.com/hashicorp/nomad/issues/2947)] * client/template: Template emits events explaining why it is blocked [[GH-3001](https://github.com/hashicorp/nomad/issues/3001)] * deployment: Disallow max_parallel of zero [[GH-3081](https://github.com/hashicorp/nomad/issues/3081)] * deployment: Emit task events explaining unhealthy allocations [[GH-3025](https://github.com/hashicorp/nomad/issues/3025)] * deployment: Better description when a deployment should auto-revert but there is no target [[GH-3024](https://github.com/hashicorp/nomad/issues/3024)] * discovery: Add HTTP header and method support to checks [[GH-3031](https://github.com/hashicorp/nomad/issues/3031)] * driver/docker: Added DNS options [[GH-2992](https://github.com/hashicorp/nomad/issues/2992)] * driver/docker: Add mount options for volumes [[GH-3021](https://github.com/hashicorp/nomad/issues/3021)] * driver/docker: Allow retry of 500 API errors to be handled by restart policies when starting a container [[GH-3073](https://github.com/hashicorp/nomad/issues/3073)] * driver/rkt: support read-only volume mounts [[GH-2883](https://github.com/hashicorp/nomad/issues/2883)] * jobspec: Add `shutdown_delay` so tasks can delay shutdown after deregistering from Consul [[GH-3043](https://github.com/hashicorp/nomad/issues/3043)] BUG FIXES: * core: Fix purging of job versions [[GH-3056](https://github.com/hashicorp/nomad/issues/3056)] * core: Fix race creating EvalFuture [[GH-3051](https://github.com/hashicorp/nomad/issues/3051)] * core: Fix panic occurring from improper bitmap size [[GH-3023](https://github.com/hashicorp/nomad/issues/3023)] * core: Fix restoration of parameterized, periodic jobs [[GH-2959](https://github.com/hashicorp/nomad/issues/2959)] * core: Fix incorrect destructive update with `distinct_property` constraint [[GH-2939](https://github.com/hashicorp/nomad/issues/2939)] * cli: Fix autocompleting global flags [[GH-2928](https://github.com/hashicorp/nomad/issues/2928)] * cli: Fix panic when using 0.6.0 cli with an older cluster [[GH-2929](https://github.com/hashicorp/nomad/issues/2929)] * cli: Fix TLS handling for alloc stats API calls [[GH-3108](https://github.com/hashicorp/nomad/issues/3108)] * client: Fix `LC_ALL=C` being set on subprocesses [[GH-3041](https://github.com/hashicorp/nomad/issues/3041)] * client/networking: Handle interfaces that only have link-local addresses while preferring globally routable addresses [[GH-3089](https://github.com/hashicorp/nomad/issues/3089)] * deployment: Fix alloc health with services/checks using interpolation [[GH-2984](https://github.com/hashicorp/nomad/issues/2984)] * discovery: Fix timeout validation for script checks [[GH-3022](https://github.com/hashicorp/nomad/issues/3022)] * driver/docker: Fix leaking plugin file used by syslog server [[GH-2937](https://github.com/hashicorp/nomad/issues/2937)] ## 0.6.0 (July 26, 2017) __BACKWARDS INCOMPATIBILITIES:__ * cli: When given a prefix that does not resolve to a particular object, commands now return exit code 1 rather than 0. IMPROVEMENTS: * core: Rolling updates based on allocation health [GH-2621, GH-2634, GH-2799] * core: New deployment object to track job updates [GH-2621, GH-2634, GH-2799] * core: Default advertise to private IP address if bind is 0.0.0.0 [[GH-2399](https://github.com/hashicorp/nomad/issues/2399)] * core: Track multiple job versions and add a stopped state for jobs [[GH-2566](https://github.com/hashicorp/nomad/issues/2566)] * core: Job updates can create canaries before beginning rolling update [GH-2621, GH-2634, GH-2799] * core: Back-pressure when evaluations are nacked and ensure scheduling progress on evaluation failures [[GH-2555](https://github.com/hashicorp/nomad/issues/2555)] * agent/config: Late binding to IP addresses using go-sockaddr/template syntax [[GH-2399](https://github.com/hashicorp/nomad/issues/2399)] * api: Add `verify_https_client` to require certificates from HTTP clients [[GH-2587](https://github.com/hashicorp/nomad/issues/2587)] * api/job: Ability to revert job to older versions [[GH-2575](https://github.com/hashicorp/nomad/issues/2575)] * cli: Autocomplete for CLI commands [[GH-2848](https://github.com/hashicorp/nomad/issues/2848)] * client: Use a random host UUID by default [[GH-2735](https://github.com/hashicorp/nomad/issues/2735)] * client: Add `NOMAD_GROUP_NAME` environment variable [[GH-2877](https://github.com/hashicorp/nomad/issues/2877)] * client: Environment variables for client DC and Region [[GH-2507](https://github.com/hashicorp/nomad/issues/2507)] * client: Hash host ID so its stable and well distributed [[GH-2541](https://github.com/hashicorp/nomad/issues/2541)] * client: GC dead allocs if total allocs > `gc_max_allocs` tunable [[GH-2636](https://github.com/hashicorp/nomad/issues/2636)] * client: Persist state using bolt-db and more efficient write patterns [[GH-2610](https://github.com/hashicorp/nomad/issues/2610)] * client: Fingerprint all routable addresses on an interface including IPv6 addresses [[GH-2536](https://github.com/hashicorp/nomad/issues/2536)] * client/artifact: Support .xz archives [[GH-2836](https://github.com/hashicorp/nomad/issues/2836)] * client/artifact: Allow specifying a go-getter mode [[GH-2781](https://github.com/hashicorp/nomad/issues/2781)] * client/artifact: Support non-Amazon S3-compatible sources [[GH-2781](https://github.com/hashicorp/nomad/issues/2781)] * client/template: Support reading env vars from templates [[GH-2654](https://github.com/hashicorp/nomad/issues/2654)] * config: Support Unix socket addresses for Consul [[GH-2622](https://github.com/hashicorp/nomad/issues/2622)] * discovery: Advertise driver-specified IP address and port [[GH-2709](https://github.com/hashicorp/nomad/issues/2709)] * discovery: Support `tls_skip_verify` for Consul HTTPS checks [[GH-2467](https://github.com/hashicorp/nomad/issues/2467)] * driver/docker: Allow specifying extra hosts [[GH-2547](https://github.com/hashicorp/nomad/issues/2547)] * driver/docker: Allow setting seccomp profiles [[GH-2658](https://github.com/hashicorp/nomad/issues/2658)] * driver/docker: Support Docker credential helpers [[GH-2651](https://github.com/hashicorp/nomad/issues/2651)] * driver/docker: Auth failures can optionally be ignored [[GH-2786](https://github.com/hashicorp/nomad/issues/2786)] * driver/docker: Add `driver.docker.bridge_ip` node attribute [[GH-2797](https://github.com/hashicorp/nomad/issues/2797)] * driver/docker: Allow setting container IP with user defined networks [[GH-2535](https://github.com/hashicorp/nomad/issues/2535)] * driver/rkt: Support `no_overlay` [[GH-2702](https://github.com/hashicorp/nomad/issues/2702)] * driver/rkt: Support `insecure_options` list [[GH-2695](https://github.com/hashicorp/nomad/issues/2695)] * server: Allow tuning of node heartbeat TTLs [[GH-2859](https://github.com/hashicorp/nomad/issues/2859)] * server/networking: Shrink dynamic port range to not overlap with majority of operating system's ephemeral port ranges to avoid port conflicts [[GH-2856](https://github.com/hashicorp/nomad/issues/2856)] BUG FIXES: * core: Protect against nil job in new allocation, avoiding panic [[GH-2592](https://github.com/hashicorp/nomad/issues/2592)] * core: System jobs should be running until explicitly stopped [[GH-2750](https://github.com/hashicorp/nomad/issues/2750)] * core: Prevent invalid job updates (eg service -> batch) [[GH-2746](https://github.com/hashicorp/nomad/issues/2746)] * client: Lookup `ip` utility on `$PATH` [[GH-2729](https://github.com/hashicorp/nomad/issues/2729)] * client: Add sticky bit to temp directory [[GH-2519](https://github.com/hashicorp/nomad/issues/2519)] * client: Shutdown task group leader before other tasks [[GH-2753](https://github.com/hashicorp/nomad/issues/2753)] * client: Include symlinks in snapshots when migrating disks [[GH-2687](https://github.com/hashicorp/nomad/issues/2687)] * client: Regression for allocation directory unix perms introduced in v0.5.6 fixed [[GH-2675](https://github.com/hashicorp/nomad/issues/2675)] * client: Client syncs allocation state with server before waiting for allocation destroy fixing a corner case in which an allocation may be blocked till destroy [[GH-2563](https://github.com/hashicorp/nomad/issues/2563)] * client: Improved state file handling and reduced write volume [[GH-2878](https://github.com/hashicorp/nomad/issues/2878)] * client/artifact: Honor netrc [[GH-2524](https://github.com/hashicorp/nomad/issues/2524)] * client/artifact: Handle tars where file in directory is listed before directory [[GH-2524](https://github.com/hashicorp/nomad/issues/2524)] * client/config: Use `cpu_total_compute` whenever it is set [[GH-2745](https://github.com/hashicorp/nomad/issues/2745)] * client/config: Respect `vault.tls_server_name` setting in consul-template [[GH-2793](https://github.com/hashicorp/nomad/issues/2793)] * driver/exec: Properly set file/dir ownership in chroots [[GH-2552](https://github.com/hashicorp/nomad/issues/2552)] * driver/docker: Fix panic in Docker driver on Windows [[GH-2614](https://github.com/hashicorp/nomad/issues/2614)] * driver/rkt: Fix env var interpolation [[GH-2777](https://github.com/hashicorp/nomad/issues/2777)] * jobspec/validation: Prevent static port conflicts [[GH-2807](https://github.com/hashicorp/nomad/issues/2807)] * server: Reject non-TLS clients when TLS enabled [[GH-2525](https://github.com/hashicorp/nomad/issues/2525)] * server: Fix a panic in plan evaluation with partial failures and all_at_once set [[GH-2544](https://github.com/hashicorp/nomad/issues/2544)] * server/periodic: Restoring periodic jobs takes launch time zone into consideration [[GH-2808](https://github.com/hashicorp/nomad/issues/2808)] * server/vault: Fix Vault Client panic when given nonexistent role [[GH-2648](https://github.com/hashicorp/nomad/issues/2648)] * telemetry: Fix merging of use node name [[GH-2762](https://github.com/hashicorp/nomad/issues/2762)] ## 0.5.6 (March 31, 2017) IMPROVEMENTS: * api: Improve log API error when task doesn't exist or hasn't started [[GH-2512](https://github.com/hashicorp/nomad/issues/2512)] * client: Improve error message when artifact downloading fails [[GH-2289](https://github.com/hashicorp/nomad/issues/2289)] * client: Track task start/finish time [[GH-2512](https://github.com/hashicorp/nomad/issues/2512)] * client/template: Access Node meta and attributes in template [[GH-2488](https://github.com/hashicorp/nomad/issues/2488)] BUG FIXES: * core: Fix periodic job state switching to dead incorrectly [[GH-2486](https://github.com/hashicorp/nomad/issues/2486)] * core: Fix dispatch of periodic job launching allocations immediately [[GH-2489](https://github.com/hashicorp/nomad/issues/2489)] * api: Fix TLS in logs and fs commands/APIs [[GH-2290](https://github.com/hashicorp/nomad/issues/2290)] * cli/plan: Fix diff alignment and remove no change DC output [[GH-2465](https://github.com/hashicorp/nomad/issues/2465)] * client: Fix panic when restarting non-running tasks [[GH-2480](https://github.com/hashicorp/nomad/issues/2480)] * client: Fix env vars when multiple tasks and ports present [[GH-2491](https://github.com/hashicorp/nomad/issues/2491)] * client: Fix `user` attribute disregarding membership of non-main group [[GH-2461](https://github.com/hashicorp/nomad/issues/2461)] * client/vault: Stop Vault token renewal on task exit [[GH-2495](https://github.com/hashicorp/nomad/issues/2495)] * driver/docker: Proper reference counting through task restarts [[GH-2484](https://github.com/hashicorp/nomad/issues/2484)] ## 0.5.5 (March 14, 2017) __BACKWARDS INCOMPATIBILITIES:__ * api: The api package definition of a Job has changed from exposing primitives to pointers to primitives to allow defaulting of unset fields. * driver/docker: The `load` configuration took an array of paths to images prior to this release. A single image is expected by the driver so this behavior has been changed to take a single path as a string. Jobs using the `load` command should update the syntax to a single string. [[GH-2361](https://github.com/hashicorp/nomad/issues/2361)] IMPROVEMENTS: * core: Handle Serf Reap event [[GH-2310](https://github.com/hashicorp/nomad/issues/2310)] * core: Update Serf and Memberlist for more reliable gossip [[GH-2255](https://github.com/hashicorp/nomad/issues/2255)] * api: API defaults missing values [[GH-2300](https://github.com/hashicorp/nomad/issues/2300)] * api: Validate the restart policy interval [[GH-2311](https://github.com/hashicorp/nomad/issues/2311)] * api: New task event for task environment setup [[GH-2302](https://github.com/hashicorp/nomad/issues/2302)] * api/cli: Add nomad operator command and API for interacting with Raft configuration [[GH-2305](https://github.com/hashicorp/nomad/issues/2305)] * cli: node-status displays enabled drivers on the node [[GH-2349](https://github.com/hashicorp/nomad/issues/2349)] * client: Apply GC related configurations properly [[GH-2273](https://github.com/hashicorp/nomad/issues/2273)] * client: Don't force uppercase meta keys in env vars [[GH-2338](https://github.com/hashicorp/nomad/issues/2338)] * client: Limit parallelism during garbage collection [[GH-2427](https://github.com/hashicorp/nomad/issues/2427)] * client: Don't exec `uname -r` for node attribute kernel.version [[GH-2380](https://github.com/hashicorp/nomad/issues/2380)] * client: Artifact support for git and hg as well as netrc support [[GH-2386](https://github.com/hashicorp/nomad/issues/2386)] * client: Add metrics to show number of allocations on in each state [[GH-2425](https://github.com/hashicorp/nomad/issues/2425)] * client: Add `NOMAD_{IP,PORT}__