Commit Graph

15000 Commits

Author SHA1 Message Date
Mahmood Ali a560f4aefc fix HashiStack eBay resource link 2019-05-02 11:19:44 -04:00
Mahmood Ali 7b72105802 Update hashicorp/go-msgpack/codec
To pick up the fix in https://github.com/hashicorp/go-msgpack/pull/8 .
2019-05-02 10:59:09 -04:00
Danielle Lancashire d824e00d1a consul: Do not deregister external checks
This commit causes sync to skip deregistering checks that are not
managed by nomad, such as service maintenance mode checks.  This is
handled in the same way as service registrations - by doing a Nomad
specific prefix match.
2019-05-02 16:54:18 +02:00
Danielle Lancashire 0b8e85118e consul: Use a stable identifier for services
The current implementation of Service Registration uses a hash of the
nomad-internal state of a service to register it with Consul, this means that
any update to the service invalidates this name and we then deregister, and
recreate the service in Consul.

While this behaviour slightly simplifies reasoning about service registration,
this becomes problematic when we add consul health checks to a service. When
the service is re-registered, so are the checks, which default to failing for
at least one check period.

This commit migrates us to using a stable identifier based on the
allocation, task, and service identifiers, and uses the difference
between the remote and local state to decide when to push updates.

It uses the existing hashing mechanic to decide when UpdateTask should
regenerate service registrations for providing to Sync, but this should
be removable as part of a future refactor.

It additionally introduces the _nomad-check- prefix for check
definitions, to allow for future allowing of consul features like
maintenance mode.
2019-05-02 16:54:18 +02:00
Yishan Lin 575814d4bb
Merge pull request #5636 from hashicorp/yishan/website-revised-resources-page
Yishan/website revised resources page
2019-05-01 18:45:47 -07:00
Yishan Lin 57302910be Cleaned spacing for syntax consistency. 2019-05-01 18:37:34 -07:00
Yishan Lin 53affa67b1 Fixed broken link to Nomad website in README 2019-05-01 11:46:03 -07:00
Yishan Lin 44f8e35586 Moved "Webinars" section below "Who Uses Nomad" 2019-05-01 11:29:22 -07:00
Mahmood Ali 914d219455 use ugorji/go/codec version that's in vendor.json 2019-05-01 12:47:14 -04:00
Mahmood Ali 21d21baf8b codegen codecs for nomad structs
`ls *[!_test].go` was ignoring any file that ends with `s.go` (or any of
the letter inside `[]`), including `structs.go`!
2019-05-01 12:42:55 -04:00
Lang Martin 29ea112586 system_sched & test cleanup comments 2019-05-01 12:25:26 -04:00
Lang Martin c490dacf76 system_sched_test extend the test to check ineligible nodes 2019-05-01 12:25:26 -04:00
Lang Martin c43bcbd35e system_sched when a node is filtered, don't mark failure 2019-05-01 12:25:26 -04:00
Lang Martin aecec5df1b system_sched_test create partially constrained job 2019-05-01 12:25:26 -04:00
Lang Martin 85dae46b51 dev/cluster meta tagged clients, raw_exec 2019-05-01 12:25:26 -04:00
Yishan Lin 0d326be223 Updated Resources Page
- Rewrote overview description
- Reformatted "Community" scetion for consistency with GitHub
- Removed IRC (redundant with Gitter)
- Added "Community Calls" section with links to prior sessions
- Added "Webinar" section with links to prior sessions
- Revised Bug Tracker copy
- Renamed "Tools for Provisioning & Experimentation" to "How to Install Nomad" section
- Bumped up the "How to Install Nomad" section right below to second on the list "Community"
- Removed "Videos" & "Blog Posts" section in favor of accurate segmentation between user talks, webinars, blog posts
- Added "Who Uses Nomad" section in consistency with new GitHub ReadME
- Removed the "Trusted By" section and disclaimer at bottom
2019-04-30 20:55:22 -07:00
Yishan Lin 15a175585e Update main tagline & boilerplate for Nomad 2019-04-30 20:51:18 -07:00
Yishan Lin 30bde7b25d
Merge pull request #5635 from hashicorp/yishan/revised-readme
Added Gitter to Resources + revised Overview in for ReadME.
2019-04-30 20:26:19 -07:00
Yishan Lin 76b4ca9c8c Added CircleCI post, Nic's two guides, and imgix Medium post into README 2019-04-30 18:42:51 -07:00
Yishan Lin 7c8b609a2b Corrected installation link via Omar's input + corrected typo 2019-04-30 13:56:13 -07:00
Yishan Lin 3e42c4067c Fixed link for Installing Nomad for Production 2019-04-30 13:39:32 -07:00
Lang Martin 9cf1c5a74d
Merge pull request #5581 from hashicorp/dev-make-boostrap-git-hooks
Dev make boostrap git hooks
2019-04-30 16:36:10 -04:00
Yishan Lin 9dcbeebbe1 Added Gitter to Resources + revised Overview for consistency. 2019-04-30 12:01:53 -07:00
Mahmood Ali c5150f81dd
Merge pull request #5621 from hashicorp/r-release-preflight
Update release/dist script with preflight
2019-04-30 14:52:03 -04:00
Mahmood Ali f47d3d5f8a add nomad streaming exec core data structures and interfaces
In this commit, we add two driver interfaces for supporting `nomad exec`
invocation:

* A high level `ExecTaskStreamingDriver`, that operates on io reader/writers.
  Drivers should prefer using this interface
* A low level `ExecTaskStreamingRawDriver` that operates on the raw stream of
  input structs; useful when a driver delegates handling to driver backend (e.g.
  across RPC/grpc).

The interfaces are optional for a driver, as `nomad exec` support is opt-in.
Existing drivers continue to compile without exec support, until their
maintainer add such support.

Furthermore, we create protobuf structures to represent exec stream entities:
`ExecTaskStreamingRequest` and `ExecTaskStreamingResponse`.  We aim to reuse the
protobuf generated code as much as possible, without translation to avoid
conversion overhead.

`ExecTaskStream` abstract fetching and sending stream entities.  It's influenced
by the grpc bi-directional stream interface, to avoid needing any adapter.  I
considered using channels, but the asynchronisity and concurrency makes buffer
reuse too complicated, which would put more pressure on GC and slows exec operation.
2019-04-30 14:02:29 -04:00
Mahmood Ali 40e62a6f17 Add ACL capabilities for nomad exec
This adds `alloc-exec` capability to allow operator to execute command into a
running task.  Furthermore, it adds `alloc-node-exec` capability, required when
the alloc task is raw_exec or a driver with no FSIsolation.
2019-04-30 14:02:16 -04:00
Mahmood Ali 7131ac3eef drivers/mock: extract command related operations
Extract command parsing and execution mocking into a separate struct.  Also,
allow mocking of different fs_isolation for testing.
2019-04-30 14:02:16 -04:00
Chris Baker a40477a7b8
test case for 5540 (#5590)
* client/metrics: modified metrics to use (updated) client copy of allocation instead of (unupdated) server copy

* updated armon/go-metrics to address race condition in DisplayMetrics
2019-04-30 10:31:35 -04:00
Lang Martin 2e643d26a2 config_parse leave the *HCL strings in place after converting times 2019-04-30 10:30:53 -04:00
Lang Martin 3ba6095fe3 config_parse_test additional config confirmation w/ sample json 2019-04-30 10:30:53 -04:00
Lang Martin fe9b31dcf9 config comment for future changes 2019-04-30 10:30:53 -04:00
Lang Martin 7e0e9b1ec1 govendor fetch github.com/hashicorp/hcl@99e2f22 2019-04-30 10:30:12 -04:00
Lang Martin 598112a1cc tag HCL bookkeeping keys with json:"-" to keep them out of the api 2019-04-30 10:29:14 -04:00
Lang Martin 43407cffe3 config_parse_test remove redundant parse direct test 2019-04-30 10:29:14 -04:00
Lang Martin b8e9c35cd0 config_parse remove unused multi-stage parsing via mapstructure 2019-04-30 10:29:14 -04:00
Lang Martin 1f86770456 config_parse_test test direct hcl parsing 2019-04-30 10:29:14 -04:00
Lang Martin 5ebae65d1a agent/config, config/* mapstructure tags -> hcl tags 2019-04-30 10:29:14 -04:00
Lang Martin 92fd988c9f config_parse add new ParseConfigFileDirectHCL
- parse by using hcl.Decode directly
- handle time.Duration strings in a second pass
- report unexpected keys in a third pass
2019-04-30 10:29:14 -04:00
Mahmood Ali fc72fff0ed test helper for registering jobs with acl
Test helper that allows registration of jobs when ACL is activated.
2019-04-30 10:23:56 -04:00
Mahmood Ali 446f06721d aux: helper method that returns token as well as ACL policy
This helper returns the token as well as the ACL policy, to be used in a later
commit for logging the token info associated with nomad exec invocation.
2019-04-30 10:23:56 -04:00
Charlie Voiselle 4c2dd47ddb
Merge pull request #5630 from jweissig/patch-5
Fixed typo
2019-04-30 08:59:16 -04:00
Mahmood Ali b18c748c41 update 0.9.1 changelog 2019-04-30 07:48:00 -04:00
Danielle 03762458ed
Merge pull request #5614 from hashicorp/dani/we-released-a-hotfix
Update CHANGELOG and version.go to reflect 0.9.1/0.9.2 split
2019-04-30 10:35:22 +02:00
Danielle Lancashire ee97edc065 Update master to reflect 0.9.1/0.9.2 split 2019-04-30 10:33:41 +02:00
Justin Weissig 4e017f5ed1
Fixed typo
Fixed typo: "dots wil be"/"dots will be".
2019-04-30 00:04:17 -07:00
Preetha Appan 6615d5c868
Add config to disable preemption for batch/service jobs 2019-04-29 18:48:07 -05:00
Yishan Lin d00da5b827
Merge pull request #5618 from hashicorp/yishan/revised-readme
yishan/revised readme
2019-04-29 15:46:20 -07:00
Yishan Lin f96866f7ee Addressed Dani's comments. 2019-04-29 10:22:27 -07:00
Preetha Appan 78c3e55198
Update 0.9.1 links in website 2019-04-29 11:46:50 -05:00
Danielle 6249f2864f
Merge pull request #5623 from jweissig/patch-4
Fixed Typo
2019-04-29 10:25:45 +02:00