When multiple developers are working on e2e testing, it helps to be
able to identify which infrastructure belongs to which Nomad SHA and
which developer. This adds tags to the EC2 instances.
This is an attempt to ease dependency management for external driver
plugins, by avoiding requiring them to compile ugorji/go generated
files. Plugin developers reported some pain with the brittleness of
ugorji/go dependency in particular, specially when using go mod, the
default go mod manager in golang 1.13.
Context
--------
Nomad uses msgpack to persist and serialize internal structs, using
ugorji/go library. As an optimization, we use ugorji/go code generation
to speedup process and aovid the relection-based slow path.
We commit these generated files in repository when we cut and tag the
release to ease reproducability and debugging old releases. Thus,
downstream projects that depend on release tag, indirectly depends on
ugorji/go generated code.
Sadly, the generated code is brittle and specific to the version of
ugorji/go being used. When go mod picks another version of ugorji/go
then nomad (go mod by default uses release according to semver),
downstream projects face compilation errors.
Interestingly, downstream projects don't commonly serialize nomad
internal structs. Drivers and device plugins use grpc instead of
msgpack for the most part. In the few cases where they use msgpag (e.g.
decoding task config), they do without codegen path as they run on
driver specific structs not the nomad internal structs. Also, the
ugorji/go serialization through reflection is generally backward
compatible (mod some ugorji/go regression bugs that get introduced every
now and then :( ).
Proposal
---------
The proposal here is to keep committing ugorji/go codec generated files
for releases but to use a go tag for them.
All nomad development through the makefile, including releasing, CI and
dev flow, has the tag enabled.
Downstream plugin projects, by default, will skip these files and life
proceed as normal for them.
The downside is that nomad developers who use generated code but avoid
using make must start passing additional go tag argument. Though this
is not a blessed configuration.
I put this property in the wrong place.
I’ve found how to fix the mock API in the tests but
they’re failing to pass with headless Chrome only,
so they’re skipped for now.
* Added Persistent Workload guide using Host Volumes
* Update website/source/guides/stateful-workloads/stateful-workloads.html.md
Co-Authored-By: Danielle <dani@hashicorp.com>
* fix client config and job spec formatting
* fix typo in description
* fix navigation for both stateful workloads guides
* show output from nomad node status to verify host volumes
* Add value prop info; info about HA
From feedback, added more information about the value proposition for
host volumes (h/t @rkettelerij), and corrected an orphaned bit from
the original guide this one was created from.
* formatting paragraphs
* remove reference to consul 1.6-beta and update nomad agent command
* remove tech preview status and update limitations
* remove beta tag in navigation
* add screenshot of count dashboard
* update example summary and remove redis references
* capitalize Consul
* minor corrections
* hcl formatting
* demo is on localhost not host ip
* clarify consul on PATH
* mention variable interpolation limitation
Splitting the immutable and mutable components of the scriptCheck led
to a bug where the environment interpolation wasn't being incorporated
into the check's ID, which caused the UpdateTTL to update for a check
ID that Consul didn't have (because our Consul client creates the ID
from the structs.ServiceCheck each time we update).
Task group services don't have access to a task environment at
creation, so their checks get registered before the check can be
interpolated. Use the original check ID so they can be updated.
