Commit graph

19314 commits

Author SHA1 Message Date
Gale Fagan c23bc0e93d
Update README.md
Fix spelling, sprinkle in assorted words
2020-07-16 19:49:31 -07:00
Gale Fagan 0c947338ab
Update README.md
Minor syntax tweaks for better secrets posture.
2020-07-16 19:34:43 -07:00
Kent 'picat' Gruber 35ca8a006f Remove TODO from open in cloud shell button 2020-07-16 16:38:54 -04:00
Kent 'picat' Gruber 6ada88f592 Remove the temporary cloudshell_git_branch query string in button
Will now use the default git branch for the repository
2020-07-16 16:38:18 -04:00
Kent 'picat' Gruber e67dada8fd Add a final Conclusion section
Includes steps to destroy the demo infrastrucure.
2020-07-16 16:34:32 -04:00
Kent 'picat' Gruber 1b56364c0a Use a full path link in the Next Steps section
This will make the link actually work when using the cloud shell.
2020-07-16 16:33:49 -04:00
Kent 'picat' Gruber 13601e0e10 Adjust the CLI section
Cleans up the wording, and points to the Next Steps section for example commands to avoid redundancy.
2020-07-16 16:33:12 -04:00
Kent 'picat' Gruber 06ee3f2094 Add project and credentials CLI vars to Terraform commands
This will automatically pass along the environment variables set in a previous step, and prevent users from manually needed to type in the values in the CLI.
2020-07-16 16:32:28 -04:00
Kent 'picat' Gruber 13eb12d29d Few minor adjustment to the Terraform steps 2020-07-16 16:24:16 -04:00
Kent 'picat' Gruber 8b77b706a8 Remove last lingering console output placeholder
So the comand is easily copyable, and consistent with the rest. However, this is probably not totally necessary in the cloud shell itself since the gcloud command will likely been initialized already.
2020-07-16 16:23:52 -04:00
Kent 'picat' Gruber 9a586404f7 Ensure all commands in terraform/gcp/README.md are easily copyable
Since this tutorial also links to a GCP cloud shell tutorial, we want to make sure all the command blocks are easily copyable to make that expierince smooth.
2020-07-16 16:17:17 -04:00
Kent 'picat' Gruber 4298646d5c Few minor adjustments to the terraform/gcp/README.md
* Adds the terraform deployment step.
* Renames the Packer-specific section.
2020-07-16 16:14:42 -04:00
Kent 'picat' Gruber 921f16f0cf Fixup open in cloud shell button query string options
* Fixes releative path to README.md tutorial, so it hopefully opens now.
* Enables only the shell now when opening the tutorial.
2020-07-16 16:10:22 -04:00
Kent 'picat' Gruber 6ede0de08f Update README.md 2020-07-16 15:58:47 -04:00
Kent 'picat' Gruber a95009a622 Update terraform/README.md 2020-07-16 15:58:42 -04:00
Kent 'picat' Gruber c8ad74b62d Create terraform/gcp/README.md 2020-07-16 15:58:15 -04:00
Kent 'picat' Gruber 9febc24d8c Create terraform/gcp/Makefile
To help make developing / testing this module much easier in the future.
2020-07-16 15:57:14 -04:00
Kent 'picat' Gruber 0a82043d8a Create terraform/gcp/env/us-east/main.tf 2020-07-16 15:56:09 -04:00
Kent 'picat' Gruber cc202e62a2 Create terraform/gcp/modules/hashistack/hashistack.tf
This module is based on the current AWS and Azure configurations, with a few tiny GCP-specific modifications:

* Defaults to only allow your Terraform's host external IP address, which can be optionally disabled. Otherwise, the UIs/APIs are exposed on the internet.
* Exposes the Vault UI and API through the load balancer.
* Uses GCP oslogin to facilitate VM SSH access, without an external SSH IP for all instances.
* Provides optional way to enable preemptible VM instances to save money.
2020-07-16 15:55:25 -04:00
Kent 'picat' Gruber 8ff9b566f1 Cleanup, enable the Vault UI, and bind to all IPv4 addresses in the config
This make the config cleaner, and allows Vault to actually be accessible through a load balancer.
2020-07-16 15:26:22 -04:00
Kent 'picat' Gruber ee3d0e0889 If $CLOUD is "gce" for auto-join, use different metadata curl command 2020-07-16 15:21:57 -04:00
Kent 'picat' Gruber a13b1a4ff6 Create terraform/shared/config/consul_gce.service 2020-07-16 15:20:34 -04:00
Kent 'picat' Gruber 0e3a7933b6 Create terraform/gcp/packer.json 2020-07-16 15:20:12 -04:00
Tim Gross fe5f5e35aa
mrd: reconcile should treat pending deployments as paused (#8446)
If a job update includes a task group that has no changes, those allocations
have their version bumped in-place. The ends up triggering an eval from
`deploymentwatcher` when it verifies their health. Although this eval is a
no-op, we were only treating pending deployments the same as paused when
the deployment was a new MRD. This means that any eval after the initial one
will kick off the deployment, and that caused pending deployments to "jump
the queue" and run ahead of schedule, breaking MRD invariants and resulting in
a state with all regions blocked.

This behavior can be replicated even in the case of job updates with no
in-place updates by patching `deploymentwatcher` to inject a spurious no-op
eval. This changeset fixes the behavior by treating pending deployments the
same as paused in all cases in the reconciler.
2020-07-16 13:00:08 -04:00
Charlie Voiselle 551ac184ee
Merge pull request #8437 from angrycub/d-reschedule-in-deploys
[docs] Rescheduling does happen during deployments
2020-07-15 15:24:21 -04:00
Mahmood Ali 647c5e4c03
Merge pull request #8435 from hashicorp/b-atomic-job-register
Atomic eval insertion with job (de-)registration
2020-07-15 13:48:07 -04:00
Michael Schurter 456e0c2e3f
Merge pull request #8441 from hashicorp/build-go1.14.5
build: update from Go 1.14.4 to Go 1.14.5
2020-07-15 10:34:15 -07:00
Mahmood Ali b6e9265d0f
Merge pull request #8383 from hashicorp/docs-security-model-followup
Revise security model feedback
2020-07-15 13:11:39 -04:00
Michael Schurter db14a4d19c build: update from Go 1.14.4 to Go 1.14.5
Go 1.14.4 contains two CVEs which are fixed in 1.14.5:

 - [CVE-2020-15586](https://golang.org/issue/34902)
 - [CVE-2020-14039](https://golang.org/issue/39360)

Upon consideration with HashiCorp security these CVEs are considered low
severity for Nomad and no new security fix binary will be released.
2020-07-15 09:49:06 -07:00
Mahmood Ali 420f244df4
Merge pull request #8436 from kneufeld/master
fixed typo in output
2020-07-15 12:18:48 -04:00
Mahmood Ali aa500f7ba3 comment compat concern in fsm.go 2020-07-15 11:23:49 -04:00
Mahmood Ali f4a921f2be no need to handle duplicate evals anymore 2020-07-15 11:14:49 -04:00
Mahmood Ali a314744210 only set args.Eval after all servers upgrade
We set the Eval field on job (de-)registration only after all servers
get upgraded, to avoid dealing with duplicate evals.
2020-07-15 11:10:57 -04:00
Mahmood Ali 910776caf0 time.Now().UTC().UnixNano() -> time.Now().UnixNano() 2020-07-15 08:49:17 -04:00
Charlie Voiselle baaab98a0d
[docs] Reschedule does happen during deployments 2020-07-14 16:29:30 -04:00
Kurt Neufeld 62851f6ccb
fixed typo in output (#1) 2020-07-14 10:33:17 -06:00
Mahmood Ali fbfe4ab1bd Atomic eval insertion with job (de-)registration
This fixes a bug where jobs may get "stuck" unprocessed that
dispropotionately affect periodic jobs around leadership transitions.
When registering a job, the job registration and the eval to process it
get applied to raft as two separate transactions; if the job
registration succeeds but eval application fails, the job may remain
unprocessed. Operators may detect such failure, when submitting a job
update and get a 500 error code, and they could retry; periodic jobs
failures are more likely to go unnoticed, and no further periodic
invocations will be processed until an operator force evaluation.

This fixes the issue by ensuring that the job registration and eval
application get persisted and processed atomically in the same raft log
entry.

Also, applies the same change to ensure atomicity in job deregistration.

Backward Compatibility

We must maintain compatibility in two scenarios: mixed clusters where a
leader can handle atomic updates but followers cannot, and a recent
cluster processes old log entries from legacy or mixed cluster mode.

To handle this constraints: ensure that the leader continue to emit the
Evaluation log entry until all servers have upgraded; also, when
processing raft logs, the servers honor evaluations found in both spots,
the Eval in job (de-)registration and the eval update entries.

When an updated server sees mix-mode behavior where an eval is inserted
into the raft log twice, it ignores the second instance.

I made one compromise in consistency in the mixed-mode scenario: servers
may disagree on the eval.CreateIndex value: the leader and updated
servers will report the job registration index while old servers will
report the index of the eval update log entry. This discripency doesn't
seem to be material - it's the eval.JobModifyIndex that matters.
2020-07-14 11:59:29 -04:00
Tim Gross bd457343de
MRD: all regions should start pending (#8433)
Deployments should wait until kicked off by `Job.Register` so that we can
assert that all regions have a scheduled deployment before starting any
region. This changeset includes the OSS fixes to support the ENT work.

`IsMultiregionStarter` has no more callers in OSS, so remove it here.
2020-07-14 10:57:37 -04:00
Tim Gross 941fca3926
changelog for MRD datacenters validation (#8429) 2020-07-13 14:03:40 -04:00
Tim Gross 0ce3c1e942
multiregion: allow empty region DCs (#8426)
It's supposed to be possible for a region not to have `datacenters` set so
that it can use the job's `datacenters` field. This requires that operators
use the same DC name across multiple regions, but that's the default client
configuration.
2020-07-13 13:34:19 -04:00
Michael Schurter 64e81f0549
Merge pull request #7042 from hashicorp/docs-healthy-deadline
docs: clarify healthy/progress_deadline relationship
2020-07-13 08:40:11 -07:00
Buck Doyle 77b4d59f5d
Change edition to Octane (#8418)
This updates the Ember edition setting to Octane, which I removed from #8319
because it required the template-only Glimmer components setting to be turned
on, which this does. These changes to templates accommodate that setting.
2020-07-13 09:26:12 -05:00
Michael Lange ce0de6e0b6
Merge pull request #8412 from hashicorp/b-ui/prefix-run-button
UI: Filter out new records from the job list page
2020-07-10 15:41:43 -07:00
Michael Lange 733151a18f Changelog addition 2020-07-10 15:31:25 -07:00
Michael Lange aaf32cabf0 Filter out new records from the job list page
When a prefix is set and the run job button is clicked,
the new job causes an error because it has no name yet.
2020-07-10 15:29:52 -07:00
Michael Lange 67d2ab73af
Merge pull request #8413 from hashicorp/b-ui/namespaces-after-token
UI: Reset the system and refetch namespaces with every token change
2020-07-10 15:29:35 -07:00
Michael Lange 069d67cbbf Changelog additions 2020-07-10 15:20:25 -07:00
Michael Lange d5b54ad84c Reset the system and refetch namespaces with every token change 2020-07-10 15:18:36 -07:00
Seth Hoenig fc194261a4
Merge pull request #8419 from hashicorp/docs-cl-vault-id-checks
docs: update changelog for vault policies lookup fix
2020-07-10 13:18:11 -05:00
Seth Hoenig 90318d9436 docs: update changelog for vault policies lookup fix 2020-07-10 13:14:12 -05:00