Commit Graph

2158 Commits

Author SHA1 Message Date
Alex Dadgar eea35626b7 Changes the relative path from joining against the alloc dir to the
task's directory.

This PR changes the behavior when given a relative host path when
mounting docker containers. Prior to this, the behavior was to mount by
joining against the alloc/ directory. This PR changes it to be against
the task/ directory.
2016-11-10 14:47:54 -08:00
Alex Dadgar e8d6227b20 Do not validate the command does not contain spaces.
This PR removes validation that the command string does not contain
spaces. This can cause issues where the path contains a folder that
includes a space ("C:\Program Files\Python35\python.exe").

Fixes #1737
2016-11-10 10:22:17 -08:00
Alex Dadgar ee921ccbb2 Merge pull request #1949 from carlpett/blacklist-fingerprints-and-drivers
Support blacklisting fingerprinters
2016-11-09 10:31:17 -08:00
Calle Pettersson 4304755c12 Address comments from PR 2016-11-09 11:50:16 +01:00
Alex Dadgar fe9a200979 Merge pull request #1952 from hashicorp/b-reserved-ports-aws
Run environmental fingerprinters after host fingerprinters and AWS overrides network
2016-11-08 15:35:46 -08:00
Alex Dadgar 20a5b6fa6b Merge pull request #1965 from hashicorp/b-docker-interpolate
Interpolate all docker driver configs that are strings
2016-11-08 15:35:27 -08:00
Alex Dadgar 3b33f49cde Merge pull request #1966 from hashicorp/b-service-interpolate
Interpolate all service/check fields
2016-11-08 15:35:19 -08:00
Alex Dadgar f1689bc7f9 Rkt env var 2016-11-08 15:14:04 -08:00
Alex Dadgar ddf101d7a2 Interpolate all check related variables 2016-11-08 14:43:46 -08:00
Alex Dadgar 691e09f863 remove debug 2016-11-08 14:21:37 -08:00
Alex Dadgar 9f2c0cb0c2 Interpolate everything that is a string 2016-11-08 14:20:51 -08:00
Diptanu Choudhury e4fdb849f9 Merge pull request #1960 from hashicorp/fix-perm-issues
Fixed permission issues on client
2016-11-08 12:57:18 -08:00
Diptanu Choudhury d9f8e3a75a Fixed comments 2016-11-08 12:55:15 -08:00
Alex Dadgar 742e11ddb4 Fix env vars relating to secretdir 2016-11-08 12:28:43 -08:00
Diptanu Choudhury 2132fbb68a Fixed permission issues on client 2016-11-08 10:57:29 -08:00
Alex Dadgar 79e55a9797 Merge pull request #1954 from hashicorp/b-secret-id
Add compatibility code for secret ID while upgrading cluster in both …
2016-11-08 09:39:52 -08:00
Calle Pettersson 8632696e2d Add blacklisting of drivers 2016-11-08 18:30:07 +01:00
Calle Pettersson b603bb007e Add blacklisting of fingerprinters 2016-11-08 18:29:44 +01:00
Alex Dadgar 9015e79aaa Add compatibility code for secret ID while upgrading cluster in both server/client mode on single nodes 2016-11-07 16:52:08 -08:00
Bastiaan Bakker 2c864172eb use snap.Alloc.TaskStates only after confirming snap.Alloc is not nil 2016-11-07 22:35:00 +01:00
Alex Dadgar 46893c7558 Merge pull request #1921 from hashicorp/f-abs-templ
Allow absolute paths for template sources
2016-11-07 12:28:49 -08:00
Alex Dadgar 92f526d902 Run environmental fingerprinters after host fingerprinters and do an override 2016-11-07 12:21:50 -08:00
Alex Dadgar 960424f086 Merge pull request #1941 from hashicorp/b-complete-transistion
Task state "dead" is terminal
2016-11-04 17:16:10 -07:00
Alex Dadgar 3643534531 Test fix 2016-11-04 17:15:58 -07:00
Alex Dadgar a9e9b61216 Merge pull request #1938 from hashicorp/b-docker-reattach
Fix Docker container creation and task runner updating
2016-11-04 17:14:40 -07:00
Alex Dadgar e6465e138b More precise marking of dead 2016-11-04 17:11:07 -07:00
Alex Dadgar 0fb7742c3c Task state "dead" is terminal 2016-11-04 16:57:24 -07:00
Michael Schurter b1a01a9a0f Add userns_mode docker config option
Fixes #1904
2016-11-04 16:53:56 -07:00
Alex Dadgar 8b7adb20e9 Fix tests 2016-11-04 15:10:18 -07:00
Alex Dadgar 0cbd0188b1 Move the wait result to bottom of methods 2016-11-04 14:58:55 -07:00
Alex Dadgar 4e8d39d674 Unique task 2016-11-04 14:53:37 -07:00
Alex Dadgar 4741a4b129 Create container much more robust 2016-11-04 14:39:56 -07:00
Alex Dadgar 2a5ac5e7ee Allow absolute paths for template sources 2016-11-02 13:04:47 -07:00
Alex Dadgar b08f4e0b97 More recoverable errors 2016-11-02 10:36:04 -07:00
Michael Schurter a0340532cb Forgot to flip a bit 2016-11-01 14:35:22 -07:00
Diptanu Choudhury 808718523b Merge pull request #1909 from hashicorp/region-wrapper
Making Nomad TLS configs region aware
2016-11-01 14:31:39 -07:00
Michael Schurter 536c2921e9 Remove ServerName because we verify based on region 2016-11-01 14:17:31 -07:00
Alex Dadgar 478eebb19b Merge pull request #1908 from hashicorp/f-vault-docs
Vault docs
2016-11-01 12:39:14 -07:00
Diptanu Choudhury 1a8fa8c8d5 Making Nomad TLS configs region aware 2016-11-01 11:55:29 -07:00
Alex Dadgar 747c3e7e68 Agent 2016-11-01 10:39:15 -07:00
Michael Schurter b84d8212ff Add LXC docs and enable by default for lxc builds 2016-11-01 10:18:16 -07:00
Diptanu Choudhury 0d911a8285 Merge pull request #1889 from hashicorp/f-migrate-snapshot-tls
Making the client use tls if the node from which migration has to be …
2016-10-31 13:08:06 -07:00
Alex Dadgar cd1791ed09 Download artifacts before templates 2016-10-31 11:29:26 -07:00
Diptanu Choudhury 4079545a92 Making the client use tls if the node from which migration has to be made has enabled tls 2016-10-31 10:20:04 -07:00
Alex Dadgar 62f1b98954 Add docker test 2016-10-28 17:53:25 -07:00
Alex Dadgar 6618f7a03d Fix passing of recoverable error from docker pull 2016-10-28 17:49:46 -07:00
Alex Dadgar c0eafb24ce Merge pull request #1887 from hashicorp/b-fixes
Fixes to Consul-Template when file exists + other small stuff
2016-10-28 17:12:31 -07:00
Alex Dadgar 893b37efc9 Merge pull request #1888 from hashicorp/b-folder-perms
Change folder permission event when not root
2016-10-28 17:12:16 -07:00
Alex Dadgar 5559300372 Change folder permission event when not root 2016-10-28 16:52:38 -07:00
Alex Dadgar b1fe7cd295 Remove thinking 2016-10-28 15:52:10 -07:00
Alex Dadgar fde7a24865 Consul-template fixes + PreviousAlloc in api 2016-10-28 15:50:35 -07:00
Michael Schurter cc115fe984 Swap log line classifiers to be consistent 2016-10-28 14:59:48 -07:00
Diptanu Choudhury f0e2178790 Making the executor lock the logger 2016-10-28 10:57:35 -07:00
Diptanu Choudhury 0d7a16bc92 Merge pull request #1879 from hashicorp/b-remote-alloc
Not blocking for remote alloc if the alloc is not sticky
2016-10-27 15:45:44 -07:00
Diptanu Choudhury 3182d0454f Adding the alloc if we can't find the TG 2016-10-27 15:45:10 -07:00
Alex Dadgar 5ddf646615 Don't delete docker images in test 2016-10-27 12:31:53 -07:00
Diptanu Choudhury 0682a1a113 Not blocking for remote alloc if the alloc is not sticky 2016-10-27 12:04:55 -07:00
Michael Schurter 43a670d442 Merge pull request #1877 from hashicorp/b-expand-docker-vols
Expand env vars in docker volume paths
2016-10-27 11:32:45 -07:00
Michael Schurter b5df74e92e Merge pull request #1862 from hashicorp/f-rkt-portmap
driver.rkt: support network configuration [waiting]
2016-10-27 11:05:55 -07:00
Michael Schurter d3835283ca Expand env vars in docker volume paths
Fixes #1876
2016-10-27 11:02:38 -07:00
Alex Dadgar 150b678a6b Merge pull request #1806 from hashicorp/f-docker4mac-fixes
A couple fixes to make Docker For Mac work
2016-10-27 09:29:40 -07:00
Michael Schurter 01b0d6746b Adjust timings to try to get test passing in Travis 2016-10-26 17:08:53 -07:00
Michael Schurter 6e1768e3ef Merge pull request #1873 from hashicorp/f-lxc-alt-build
Put lxc support behind a flag
2016-10-26 16:05:32 -07:00
Diptanu Choudhury 6c527581af Merge pull request #1871 from hashicorp/b-tls-tests
Fixed a bunch of TLS related failures
2016-10-26 15:19:54 -07:00
Diptanu Choudhury 1098dc4aa3 Fixed alloc dir move tests 2016-10-26 15:17:57 -07:00
Alex Dadgar a309fd6ba3 Fix Validate_UserEnforcement 2016-10-26 15:03:41 -07:00
Michael Schurter f430203505 Put lxc support behind a flag
Since lxc support requires linking to a C lib at compile and runtime
I'm putting it behind a build flag to avoid forcing all nomad users to
install liblxc (lxc-dev for development).
2016-10-26 14:55:54 -07:00
Diptanu Choudhury 84722234b4 Fixed a bunch of TLS related failures 2016-10-26 14:08:46 -07:00
Diptanu Choudhury 067fcda3fe Making the cli use TLS if the client has enabled TLS 2016-10-26 11:13:53 -07:00
Michael Schurter 98600b5943 Fix LXC driver interface impls 2016-10-26 10:06:03 -07:00
Alex Dadgar 3c4a27e72b Fix panic 2016-10-25 17:27:13 -07:00
Michael Schurter 21b8030ded Fix tests for rkt port map 2016-10-25 17:06:41 -07:00
Diptanu Choudhury 50ca5e1e9d Merge pull request #1853 from hashicorp/f-rpc-http-tls
TLS support for http and RPC
2016-10-25 16:14:43 -07:00
Diptanu Choudhury 7c61e115bd Moved tlsutil into helpers 2016-10-25 16:05:37 -07:00
Diptanu Choudhury 353e7fc7f1 Moving the certs into tlsutil package 2016-10-25 16:01:53 -07:00
Diptanu Choudhury cf35aeac84 Moving the TLSConfig to structs 2016-10-25 15:57:38 -07:00
Kenjiro Nakayama 18bbfe8325 driver.rkt: add rkt drriver port mapping test 2016-10-25 15:41:49 -07:00
Kenjiro Nakayama 56d91632ca driver.rkt: support port mapping with net and port options 2016-10-25 15:41:49 -07:00
Diptanu Choudhury 9d3cdded9a Merge pull request #1699 from hashicorp/f-lxc-driver
LXC Support
2016-10-25 15:17:44 -07:00
Diptanu Choudhury b5cc153d54 Added lxc related dependencies 2016-10-25 15:17:02 -07:00
Alex Dadgar 9bba036f13 Merge pull request #1852 from hashicorp/b-service-validation
Interpolate and then validate services
2016-10-25 14:28:32 -07:00
Alex Dadgar 4082732d3a Interpolate and then validate services 2016-10-25 14:27:49 -07:00
Alex Dadgar bf0981363f Merge pull request #1850 from hashicorp/f-fs-secret
Disallow fs to read secret directory
2016-10-25 11:35:08 -07:00
Alex Dadgar 8e07c2750e Merge pull request #1839 from hashicorp/f-signal-constraints
Signal creates an auto-constraints
2016-10-25 11:09:33 -07:00
Alex Dadgar f8419fdd6e Add CaPath to Vault config in consul-template 2016-10-25 11:01:50 -07:00
Diptanu Choudhury eefc8db3b3 Enabling TLS on cli 2016-10-25 10:39:17 -07:00
Michael Schurter 4f45aece4b Fingerprint rkt volume support and make periodic
Fix rkt docs and custom volume mounting
2016-10-25 09:46:49 -07:00
Michael Schurter 5d358c7eba Allow mounting alloc-dir-relative paths in docker 2016-10-25 09:46:49 -07:00
Michael Schurter 49ed6da0ad Enable rkt and docker volume mounting by default 2016-10-25 09:46:49 -07:00
Michael Schurter f075bda9b9 Make volume name unique 2016-10-25 09:46:49 -07:00
Michael Schurter 83a11fc93b Bump minimum required rkt version; update docs
Make section names match between docker and rkt
2016-10-25 09:46:49 -07:00
Michael Schurter edf657b58a Fix docker reference in rkt test 2016-10-25 09:46:49 -07:00
Michael Schurter 02ed35bd1c Add arbitrary volume support to rkt 2016-10-25 09:46:49 -07:00
Michael Schurter 473c28824c Fix standard mounts in rkt and tests 2016-10-25 09:46:49 -07:00
Alex Dadgar da8b05ba17 Fix merge 2016-10-24 17:04:10 -07:00
Alex Dadgar 03eba049ed Merge pull request #1848 from hashicorp/f-vault-error
Thread through whether DeriveToken error is recoverable or not
2016-10-24 15:01:18 -07:00
Diptanu Choudhury e03927bb5c Changed the way TLS config is parsed 2016-10-24 13:56:19 -07:00
Alex Dadgar e85d0ebace Merge pull request #1840 from hashicorp/f-kill-fail
Change how we mark tasks as failed and allow consul-template to fail tasks
2016-10-24 13:40:52 -07:00
Alex Dadgar 4ae735c8ba Disallow fs to read secret directory 2016-10-24 11:14:05 -07:00
Alex Dadgar 692a809919 Merge pull request #1842 from hashicorp/f-version-and-id
Print the version and client node ID
2016-10-24 10:13:33 -07:00
Diptanu Choudhury 2e3118e69c Implemented TLS support for http and rpc 2016-10-23 22:22:00 -07:00
Alex Dadgar 5577e53b20 Merge pull request #1845 from hashicorp/f-remove-disk-usage-acct
Remove disk usage enforcement
2016-10-22 19:01:51 -07:00
Alex Dadgar e85a67a49a Fix signal test for docker 2016-10-22 18:32:48 -07:00
Alex Dadgar ede3a814ba Small fixes 2016-10-22 18:20:50 -07:00
Alex Dadgar 0070178741 Thread through whether DeriveToken error is recoverable or not 2016-10-22 18:08:30 -07:00
Michael Schurter d937d3aede Fix comment form 2016-10-21 16:56:33 -07:00
Michael Schurter 285e80ac0f Remove disk usage enforcement
Many thanks to @iverberk for the original PR (#1609), but we ended up
not wanting to ship this implementation with 0.5.

We'll come back to it after 0.5 and hopefully find a way to leverage
filesystem accounting and quotas, so we can skip the expensive polling.
2016-10-21 13:55:51 -07:00
Alex Dadgar aa0d8d0d8d Print the version and client node ID 2016-10-20 17:46:04 -07:00
Alex Dadgar 46a7d1a0d7 Change how we mark tasks as failed and allow consul-template to fail tasks 2016-10-20 17:27:16 -07:00
Alex Dadgar 41b5679015 Advertise signalling abilities 2016-10-19 15:06:23 -07:00
Alex Dadgar ae1ea0e5ba Actually mount the local directory 2016-10-18 15:57:12 -07:00
Alex Dadgar b384bff053 Feedback 2016-10-18 15:01:04 -07:00
Alex Dadgar ba0b3963ef Comments 2016-10-18 11:36:04 -07:00
Alex Dadgar 4f8bfd7b18 Tests 2016-10-18 11:24:20 -07:00
Alex Dadgar 36cfe6e89e Large refactor of task runner and Vault token rehandling 2016-10-18 11:24:20 -07:00
Alex Dadgar 53eeec9bc1 Merge pull request #1801 from hashicorp/f-signals
Consul-template signal change mode
2016-10-18 11:23:47 -07:00
Michael Schurter 34f7cbd10f Disable lxc by default 2016-10-13 13:22:12 -07:00
Michael Schurter 1dbb2b7164 Cleanup comments/whitespace 2016-10-13 13:05:55 -07:00
Michael Schurter 38b2020291 Mount secret dir 2016-10-13 12:45:33 -07:00
Diptanu Choudhury 4e86a5f906 throwing an error if stats line can't be converted to k/v pair 2016-10-12 17:18:58 -07:00
Diptanu Choudhury ea5d9d959a Bind mounting alloc dir into container 2016-10-12 17:18:58 -07:00
Diptanu Choudhury 6312ea3f8f Setting the network type 2016-10-12 17:18:58 -07:00
Diptanu Choudhury ce334e0e04 Adding cpu resource limits 2016-10-12 17:18:58 -07:00
Diptanu Choudhury bb2a580ef1 Implemented an LXC Driver 2016-10-12 17:18:58 -07:00
Evan Phoenix e7a98d5500 Make EvalSymlink errors more verbose 2016-10-12 17:07:21 -07:00
Evan Phoenix 8864a506aa Disable the syslog logging system on Docker For Mac
The syslog logging system depends on the ability for a unix socket to be
accessed by the docker daemon in the $TMPDIR of the host. This doesn't
work on Docker For Mac because the docker daemon is running inside a VM,
and while /tmp is accessible, the filesystem used to share them doesn't
support unix socket files, and thus it doesn't work.
2016-10-12 17:07:21 -07:00
Evan Phoenix f8a65a3b9d Resolve alloc/state directories to make Docker For Mac happy
* In -dev mode, `ioutil.TempDir` is used for the alloc and state
directories.
* `TempDir` uses `$TMPDIR`, which os OS X contains a per user
directory which is under `/var/folder`.
* `/var` is actually a symlink to `/private/var`
* Docker For Mac validates the directories that are passed to bind and on
OS X. That whitelist contains `/private`, but not `/var`. It does not
expand the path, and so any paths in `$TMPDIR` fail the whitelist check.

And thusly, by expanding the alloc/state directories the value passed
for binding does contain `/private` and Docker For Mac is happy.
2016-10-12 17:06:25 -07:00
Alex Dadgar eec1a154ec add plugin kill 2016-10-12 13:24:22 -07:00
Alex Dadgar 86238387e7 Send Executor Ctx separately 2016-10-12 11:35:29 -07:00
Alex Dadgar db4e676d73 Merge pull request #1803 from hashicorp/b-vault-parse
Fix Vault Config parsing of booleans
2016-10-11 13:47:46 -07:00
Michael Schurter ed5ff3a104 Merge pull request #1804 from hashicorp/f-job-env-var
Add NOMAD_JOB_NAME to task environment
2016-10-11 13:34:54 -07:00
Alex Dadgar 82960c46d8 Tests 2016-10-11 13:28:18 -07:00
Ben Barnard 83f647ed84 Replace "the the" with "the" in documentation and comments 2016-10-11 15:31:40 -04:00
Michael Schurter ca5439eca1 Add NOMAD_JOB_NAME to environment 2016-10-11 11:20:42 -07:00
Alex Dadgar 751aa114bf Fix Vault parsing of booleans 2016-10-10 18:04:39 -07:00
Alex Dadgar bc35eaee21 Task runner sends signals 2016-10-10 15:09:00 -07:00
Alex Dadgar 00a1234c55 Executor + Java/Raw Exec/Exec 2016-10-10 11:47:04 -07:00
Alex Dadgar 5b01b1be1b Rkt 2016-10-10 11:47:04 -07:00
Alex Dadgar 280af8f4d1 Docker + Qemu 2016-10-10 11:47:04 -07:00
Alex Dadgar 08eeef0140 Merge pull request #1796 from hashicorp/f-task-runner
Task runner integrates with TaskTemplateManager
2016-10-07 15:13:14 -07:00
Michael Schurter d8f8048d85 Merge pull request #1767 from hashicorp/f-docker-volumes-logging
Support Docker Volumes and Logging
2016-10-07 12:10:59 -07:00
Michael Schurter f0d04bd798 Add comment and fix log line code style 2016-10-07 11:58:21 -07:00
Michael Schurter 523dbfcc81 Remove VolumesFrom feature
Since containers are named with alloc ids it's difficult to use safely.
Not to mention task scheduling ordering issues could break it as well.
2016-10-07 11:58:13 -07:00
Alex Dadgar e2d49eb4a2 Comments 2016-10-06 15:21:59 -07:00
Alex Dadgar 68c5fe78f8 Tests 2016-10-06 15:17:34 -07:00
Alex Dadgar 8fb07bb083 Fix handling of restart in TaskEvents 2016-10-06 15:06:54 -07:00
Alex Dadgar 8eb7fa91cf Start of integration 2016-10-06 15:05:49 -07:00
Alex Dadgar 3693de99d8 Merge pull request #1783 from hashicorp/f-consul-template
Consul template manager
2016-10-06 15:05:01 -07:00
Alex Dadgar c7f76ea78d Make tests channel based 2016-10-06 14:51:54 -07:00
Alex Dadgar 19a6aefd68 more vendoring 2016-10-06 12:36:44 -07:00
Michael Schurter f777faba00 Add comments to config key constants 2016-10-03 16:04:33 -07:00
Michael Schurter 0d66b8aef0 Only launch syslog server if container uses syslog 2016-10-03 15:22:10 -07:00
Michael Schurter 44219cc083 Put docker volume support behind conf flag
Also add tests and fix bug with logging driver configuration.
2016-10-03 15:02:50 -07:00
Jan-Hendrik Lendholt a26a501120 Fixed a bug when giving in another logging driver than syslog.
Before this commit, if the Logging config did not contain a logging option "syslog-address", it would definitely insert this option.
If then, you decide to take another logdriver than syslog, docker would fail because it received a wrong log option for the selected driver.
Now, nomad will only insert the syslog address in a hard way if there are no logging options at all - this way it keeps the default nomad settings.
2016-10-03 15:02:50 -07:00
Jan-Hendrik Lendholt 6c7cbe5fcb Added support to mount host folders into container. For example if you don't want to bake certificates into the container, you can mount them into the directory directly.
Furthermore, I added support for volumes-from.

Currently, there is no support to move the data from one container to another, hence: If a container spawns on another host, it is very likely, that the data will not be found.
2016-10-03 15:02:49 -07:00
Jan-Hendrik Lendholt ac5cde4641 Added logging options support for docker driver 2016-10-03 15:02:49 -07:00
Alex Dadgar d2837dec44 Do not allow path to escape the alloc dir for the FS commands 2016-10-03 14:58:44 -07:00
Michael Schurter 6dea6df919 Restore lost chan inits 2016-10-03 14:56:50 -07:00
Alex Dadgar 4eaabd675c Consul Template Manager 2016-10-03 12:59:31 -07:00
Diptanu Choudhury d50c395421 Getting snapshot of allocation from remote node (#1741)
* Added the alloc dir move

* Moving allocdirs when starting allocations

* Added the migrate flag to ephemeral disk

* Stopping migration if the allocation doesn't need migration any more

* Added the GetAllocDir method

* refactored code

* Added a test for alloc runner

* Incorporated review comments
2016-10-03 09:59:57 -07:00
Michael Schurter b117725dc9 Only log consul errors once since last succesful run 2016-09-28 17:18:45 -07:00
Alex Dadgar 9beee9c891 Merge pull request #1762 from hashicorp/b-scan-pids
Constant size space tracking of pids
2016-09-27 17:28:35 -07:00
Alex Dadgar 320b89d57a Constant time space tracking of pids 2016-09-27 16:57:26 -07:00
Michael Schurter 80085ddda5 Merge pull request #1735 from hashicorp/b-bootstrap-flapping
Retry all servers on RPC call failure
2016-09-27 16:33:15 -07:00
Michael Schurter d486de3804 Remove unused const 2016-09-27 16:04:01 -07:00
Diptanu Choudhury 2b1d214b0d Avoiding copying files if they are already present in chrootw (#1753) 2016-09-27 11:43:27 -07:00
Michael Schurter 2e696c5e61 Fix lies found in comments by fact checkers 2016-09-26 16:51:53 -07:00
Michael Schurter 11cf9686a6 No need to put reaper ticker on the struct 2016-09-26 16:15:19 -07:00
Michael Schurter 2eb0062959 Drop clumsy timeout on discovery notifications
It's better to just let goroutines fallback to their longer retry
intervals then try to be clever here.
2016-09-26 16:05:21 -07:00
Michael Schurter 307e674eca Flip disco chan; clarify method names/comments 2016-09-26 15:52:40 -07:00
Michael Schurter 888ee21270 Return csv of servers from Stats, not just count 2016-09-26 15:40:26 -07:00
Alex Dadgar b28e817b1d Test fix 2016-09-26 15:35:59 -07:00
Michael Schurter 7dc0079dd2 doDisco -> triggerDiscoveryCh; discovered -> serversDiscoveredCh
Also fix log line formatting
2016-09-26 15:21:28 -07:00
Michael Schurter 434e4be97c noServers -> noServersErr 2016-09-26 15:12:35 -07:00
Michael Schurter b2ddb85a78 consul -> Consul 2016-09-26 15:06:57 -07:00
Diptanu Choudhury 12c7873db2 Closing files when files are removed 2016-09-23 22:17:53 -07:00
Michael Schurter 37cfb2769c Replace periodic handlers with event driven disco
Remove use of periodic consul handlers in the client and just use
goroutines. Consul Discovery is now triggered with a chan instead of
using a timer and deadline to trigger.

Once discovery is complete a chan is ticked so all goroutines waiting
for servers will run.

Should speed up bootstraping and recovery while decreasing spinning on
timers.
2016-09-23 17:02:48 -07:00
Michael Schurter 2ab5264595 Retry all servers on RPC call failure
rpcproxy is refactored into serverlist which prioritizes good servers
over servers in a remote DC or who have had a failure.

Registration, heartbeating, and alloc status updating will retry faster
when new servers are discovered.

Consul discovery will be retried more quickly when no servers are
available (eg on startup or an outage).
2016-09-23 11:44:48 -07:00
Diptanu Choudhury 589356fd55 Adding a snapshot endpoint on the client (#1730) 2016-09-21 21:28:12 -07:00
Alex Dadgar 12de69a66f Struct and parse 2016-09-21 11:31:09 -07:00
Alex Dadgar 50efdb00e9 Merge pull request #1713 from hashicorp/f-alloc-runner-vault
Vault integration in client
2016-09-20 16:15:55 -07:00
Alex Dadgar 64de46432a Merge pull request #1677 from hashicorp/f-vault-implicit-constraint
Vault implicit Task Group constraint + allow root tokens
2016-09-20 16:15:32 -07:00
Diptanu Choudhury f7a9b39e8c Ensuring that we are not emitting stats when handle is nil (#1723)
* Ensuring that we are not emitting stats when handle is nil

* Updated the changelog
2016-09-20 11:29:34 -07:00
Alex Dadgar 83905075e5 Fix comment 2016-09-17 11:31:17 -07:00
Alex Dadgar 40fc1d1dfd Task runner test 2016-09-15 17:39:08 -07:00
Alex Dadgar 0fefaef008 Alloc runner tests 2016-09-15 17:24:09 -07:00
Alex Dadgar 0f40bd41a3 Handle recovery failure 2016-09-15 12:50:44 -07:00
Alex Dadgar 688e616200 Fix token renewal 2016-09-15 11:20:51 -07:00
Alex Dadgar ec152a6d12 Clean up vault client 2016-09-14 18:10:56 -07:00
Alex Dadgar 6702a29071 Vault token threaded 2016-09-14 13:30:01 -07:00
Robert Neumayer 8dc19dbd10 Log adding of servers at INFO level 2016-09-14 22:24:17 +02:00
Michael Schurter 9a2a17b48f Merge pull request #1682 from hashicorp/b-sanity-check-state-file
Prevent state file corruption and check state file sanity on save
2016-09-13 11:24:34 -07:00
Michael Schurter cd8606b9e3 Revert "A nil context isn't an error"
This reverts commit fe9fe4c26259c1ad3bd7e94bd711418aaf819b20.
2016-09-12 12:56:12 -07:00
Diptanu Choudhury ab6a9a5120 Fixing alloc runner tests 2016-09-04 19:09:08 -07:00
Diptanu Choudhury 0f77d81f7d Merge pull request #1683 from mwieczorek/enable-syslog-for-windows
Enable syslog for windows
2016-09-04 17:07:25 -07:00
Michal Wieczorek 94071fc294 Enable syslog server and universal collector for windows 2016-09-05 00:26:36 +02:00
Michael Schurter 8a57913a44 A nil context isn't an error 2016-09-02 16:24:53 -07:00
Michael Schurter f601361d58 Don't serialize task states twice in state files 2016-09-02 16:07:06 -07:00
Michael Schurter 6cb6d9cdf1 Lock around saving state
Prevent interleaving state syncs as it could conceivably lead to
empty state files as per #1367
2016-09-02 16:07:06 -07:00
Michael Schurter e7dd443447 Add sanity check to SaveState
Also just reuse the task states snapshot taken by `Alloc()` instead of
doing a redundant copy.
2016-09-02 16:07:06 -07:00
Alex Dadgar eecef73302 syscall error 2016-09-02 15:00:46 -07:00
Alex Dadgar eef786dd9d Secret dir materialized in alloc/task directory 2016-09-02 12:44:05 -07:00
Alex Dadgar 2c8dd8bbd3 Revert "Introduce a Secret/ directory" 2016-09-01 17:23:15 -07:00
Alex Dadgar 4a8fba5cf7 small fixes 2016-09-01 13:38:31 -07:00
Alex Dadgar b0adaa5301 Allow root token 2016-09-01 12:05:08 -07:00
Alex Dadgar 8ca3a16825 Fingerprint 2016-09-01 11:10:14 -07:00
Alex Dadgar 1ed454dd60 Merge pull request #1671 from hashicorp/f-secret-dir2
Introduce a Secret/ directory
2016-09-01 09:56:17 -07:00
Alex Dadgar 9fa23e3536 Symlink on windows 2016-08-31 21:41:44 -07:00
Alex Dadgar 5d3b47e648 Address comments and reserve 2016-08-31 18:11:02 -07:00
Michael Schurter cbb0b8e31e Merge pull request #1668 from hashicorp/b-fix-consul-updates
Fix old services not getting removed from consul on update
2016-08-31 17:17:09 -07:00
Alex Dadgar 0626eb9619 environment variables 2016-08-31 13:56:11 -07:00
Alex Dadgar d59e14eed4 Interface + tests 2016-08-30 21:40:32 -07:00
Vishal Nayak b6b73545ea Merge pull request #1606 from hashicorp/f-vault-client
VaultClient for Nomad client's interactions with Vault
2016-08-30 13:13:54 -04:00
vishalnayak d0ad1603c3 Print debug message only when error is non-nil 2016-08-30 13:14:34 -04:00
vishalnayak 55a6f06e15 Addressed review feedback 2016-08-30 13:08:13 -04:00
vishalnayak 3808dd0ff8 Return only fatal error to renewal error channel 2016-08-30 12:46:59 -04:00
vishalnayak a0dbfe25b3 Fix tests 2016-08-29 21:30:06 -04:00
vishalnayak 82f6209e97 tokenDeriver function pointer to derive tokens.
Remove rpc*, connPool, node and region from vaultclient.
2016-08-29 20:32:05 -04:00
Alex Dadgar 14b7126511 Secret dir, hello world 2016-08-29 15:41:52 -07:00
vishalnayak f35bb409b6 Use Job.LookupTaskGroup 2016-08-29 16:34:39 -04:00
vishalnayak 160ba48eb4 Address review feedback 2016-08-29 12:47:33 -04:00
Alex Dadgar aaca0bdaf4 Make maxSize exported so that it is serialized 2016-08-28 17:48:35 -07:00
Michael Schurter 32626ac608 Remove unused embedded lock on ExecContext 2016-08-26 11:58:03 -07:00
Michael Schurter d31f373a5b Merge pull request #1653 from hashicorp/b-fix-artifact-retry
Don't fail other tasks when retrying artifact get
2016-08-26 09:53:39 -07:00
Michael Schurter 592b898f43 Make sure bad doesn't fail before web runs 2016-08-25 17:25:51 -07:00
Michael Schurter 139d2e7939 Ensure bad task reaches terminal state within test 2016-08-25 16:05:19 -07:00
Michael Schurter a739058fa3 Ensure web task exited successfully
Web task should run to completion successfully while the `bad` task is
retrying artifact downloads.
2016-08-25 14:42:50 -07:00
Michael Schurter 5ce26f82fe Don't fail other tasks when retrying artifact get
The artifact fetching may be retried and succeed, so don't set the task
as dead.

Fixes #1558
2016-08-25 13:16:41 -07:00
Ivo Verberk d89dc40fe2 Small comment fix. 2016-08-25 20:50:11 +02:00
Ivo Verberk 57012e8d8c Monitor the complete alloc directory, not just the shared part. 2016-08-25 20:48:19 +02:00
Ivo Verberk 9113244131 Don't duplicate TaskKilled event and check for TaskSiblingFailed. 2016-08-25 20:11:10 +02:00
Diptanu Choudhury e9b27a528c Fixed the raw_exec fingerprint test 2016-08-24 13:38:43 -05:00
Diptanu Choudhury 10e10b2fe1 fixed the qemu fingerprinter test 2016-08-24 12:25:02 -05:00
vishalnayak 56e42cf03d Employ DeriveVaultToken API and flesh-up DeriveToken 2016-08-24 12:29:59 -04:00
vishalnayak 6002e596c4 VaultClient for Nomad Client 2016-08-24 09:43:45 -04:00
Diptanu Choudhury 9309247f9d Fixed a java test 2016-08-23 16:51:09 -05:00
Diptanu Choudhury 05fe72e89e fixed the exec fingerprinter test 2016-08-23 16:40:56 -05:00
Diptanu Choudhury 588a4802c1 removing driver_mock 2016-08-23 16:05:03 -05:00
Alex Dadgar 0f3ec9c759 Fix TestDockerDriver_Fingerprint 2016-08-23 10:39:40 -07:00
Alex Dadgar 1da8566322 Merge pull request #1580 from hashicorp/f-disk-usage-monitoring
Monitor and enforce shared allocation directory disk usage
2016-08-23 09:49:53 -07:00
Diptanu Choudhury 1e1eef56a1 Putting the mock driver behind a build flag 2016-08-22 15:02:28 -05:00
Diptanu Choudhury 4ca623bcfe blocking chained allocations until previous allocation hasn't terminated 2016-08-22 11:34:24 -05:00
Kenjiro Nakayama b06c6d9311 driver.docker: tiny: debug messages output task name instead of image name 2016-08-21 19:51:32 +09:00
Alex Dadgar a90dafe9ab handle the upgrade case 2016-08-18 19:01:24 -07:00
Ivo Verberk 2a17895a83 Disk resource monitoring and enforcement 2016-08-18 07:59:03 +02:00
Alex Dadgar 895c31f605 Nodes generate Secret ID and used for retrieving allocations and registering 2016-08-17 16:31:47 -07:00
vishalnayak ff9bb5b08b Disable Vault instead of supplying test token 2016-08-17 16:25:38 -07:00
vishalnayak 1c1457b01b Fix a few client tests 2016-08-17 16:25:38 -07:00
Alex Dadgar 7d899b6c60 Pass Vault config to client 2016-08-17 16:23:29 -07:00
Diptanu Choudhury dc0e395982 re-using copyimage 2016-08-17 15:25:03 -07:00
Diptanu Choudhury 0d7cd53c63 Fixed docker tests 2016-08-17 15:25:03 -07:00
Diptanu Choudhury ab7f8847c1 changing error statement 2016-08-17 13:48:31 -07:00
Diptanu Choudhury 2968ce9399 Fixed the docker script check 2016-08-17 13:23:48 -07:00
Alex Dadgar 3b9188fcf0 Merge pull request #1598 from nak3/rkt-fix5
driver.rkt: Remove unnecessary job validation
2016-08-16 15:12:45 -07:00
Alex Dadgar 423da3e99d Merge pull request #1593 from hashicorp/f-rereg
Reregister Client on failed heartbeat
2016-08-16 13:29:34 -07:00
Kenjiro Nakayama c97beb8deb driver.rkt: Remove unnecessary job validation 2016-08-16 23:33:34 +09:00
ramukima 4f18963a97 go fmt performed code when copied from another directory got messed up again ? Ok, ran go fmt again 2016-08-16 09:20:13 -04:00
ramukima 17b902f20f issue-1588 : Allow extra driver config args as a passthrough for qemu executable from a task specification 2016-08-15 23:36:13 -04:00
Alex Dadgar 84820db86f If the client detects that a heartbeat has failed because it is not registered, reregister 2016-08-15 17:24:09 -07:00
Alex Dadgar 64b86bed0d Merge pull request #1581 from nak3/fix-rkt2
Set host environment variables to taskEnv of rkt driver
2016-08-15 10:31:10 -07:00
Alex Dadgar a2b603bd82 Merge pull request #1586 from nak3/rkt-fix3
tiny: Catch error returned from SyncServices in rkt driver
2016-08-15 10:29:28 -07:00
Kenjiro Nakayama d0ccdacd08 rkt.driver: Fix wrong MB calculation 2016-08-14 14:27:42 +09:00
Kenjiro Nakayama 906526fbe0 tiny: Catch error returned from SyncServices in rkt driver 2016-08-14 13:11:17 +09:00
Kenjiro Nakayama 0efc0e0231 Set host environment variables to taskEnv of rkt driver 2016-08-14 00:42:53 +09:00
Diptanu Choudhury 8c8d00dd5b Merge pull request #1572 from hashicorp/fix-docker-test
Fixed docker tests
2016-08-12 13:04:33 -07:00
Diptanu Choudhury f3420c65e3 Updated the busybox images 2016-08-12 11:39:58 -07:00
Diptanu Choudhury dd7e69006e Not running tests parallal 2016-08-11 21:53:27 -07:00
Alex Dadgar b34d65f1a5 Update test.sh 2016-08-11 21:35:50 -07:00
Diptanu Choudhury 839ecd1df6 Fixed docker tests 2016-08-11 19:28:41 -07:00
Alex Dadgar ed23dff23c Merge pull request #1571 from hashicorp/t-logs-tests
Fixing logs tests on travis
2016-08-11 19:24:46 -07:00
Alex Dadgar b3dca54a1a Wrap file rotator tests in wait for 2016-08-11 19:23:03 -07:00
Alex Dadgar 8323b6a0b5 only use polling 2016-08-11 18:59:48 -07:00
Diptanu Choudhury b71f687f62 Merge pull request #1413 from hashicorp/b-tests
Fix flaky tests
2016-08-11 18:15:29 -07:00
Alex Dadgar 2614b0f06e Update task_runner_test.go 2016-08-11 18:01:27 -07:00
Alex Dadgar ac7749f70e Fix task runner test 2016-08-11 13:16:17 -07:00
Kenjiro Nakayama fe13453012 Update after the review 2016-08-11 10:53:33 +09:00
Kenjiro Nakayama c3b871e90d Return error when client failed to collect host stats 2016-08-11 09:38:28 +09:00
Diptanu Choudhury 9a75052d2c Merge pull request #1518 from pubnub/feature/chroot-map-rebase
Add config field to specify chroot mapping for exec driver
2016-08-10 17:00:03 -07:00
Diptanu Choudhury 28b3f511e0 Fixed some error messages 2016-08-10 15:17:32 -07:00
Diptanu Choudhury 1b74f863fe Merge pull request #1533 from nak3/fix-error-in-client
tiny: Return fmt.Errorf instead of duplicated error messages
2016-08-10 15:13:47 -07:00
Kenjiro Nakayama 6ec6c27cb4 Update debug option from string to bool 2016-08-09 16:51:00 +09:00
Kenjiro Nakayama 11a8a7218e Add debug option to rkt task config 2016-08-09 09:01:05 +09:00
Kenjiro Nakayama 6a810e6f1e Update after review 2016-08-09 08:57:26 +09:00
Kenjiro Nakayama 5c621b74e5 tiny: Return fmt.Errorf instead of duplicated error messages 2016-08-09 08:57:26 +09:00
Jay Oster 09113ffbc8 Fix Linux executor isolation test
- Properly expects the hard-coded mounts (alloc, dev, and proc) and hardcoded local directories (local and tmp)
- Also verifies that etc contains only the requested paths
2016-08-08 14:04:09 -07:00
Diptanu Choudhury c0ec1b2101 Merge pull request #1532 from nak3/fix-fingerprint-cpu-log
tiny: Fix duplicated error message in CPU fingerprint
2016-08-08 13:44:49 -04:00
Diptanu Choudhury 70d2f8ef1d Merge pull request #1534 from nak3/fix-intask_runner
tiny: print task name and error message for SaveState error
2016-08-08 13:37:25 -04:00
Kenjiro Nakayama e7863ea8ee tiny: print task name and error message for the SaveState error in task_runner 2016-08-07 13:33:58 +09:00
Kenjiro Nakayama 71371fc592 tiny: Fix duplicated error message in CPU fingerprint 2016-08-07 12:49:40 +09:00
Kenjiro Nakayama 60b58eed84 Update GetArtifact by removing unused logger 2016-08-06 23:37:32 +09:00
Diptanu Choudhury fb178a1f5f Merge pull request #1477 from nak3/add-syslog_server-test
Add Syslog server start shutdown test
2016-08-05 12:01:24 -07:00
Alex Dadgar ddd8adce96 changelog + use driver config 2016-08-05 10:55:20 -07:00
Alex Dadgar f8e4bc73c9 Merge pull request #1493 from nak3/rkt-fix1
Pass command and trust_prefix to the validation of rkt task configuration
2016-08-05 10:52:02 -07:00
Alex Dadgar 096956257d changelog 2016-08-05 10:47:44 -07:00
Kenjiro Nakayama fc1195b7ea Add Syslog server start shutdown test 2016-08-06 02:01:33 +09:00
Michal Wieczorek b688261a99 Set windows containers default network mode to 'nat' 2016-08-05 06:01:26 +02:00
Jay Oster 2ae059b41d Address review comments
- Simplify map length check in Linux Executor
- Added a `chroot_env` test for config parser
- Moved `ChrootEnv` field from ExecutorCommand to ExecutorContext
- Added a test for `chroot_env` functionality
2016-08-04 15:33:06 -07:00
Diptanu Choudhury 5ff750db96 Merge pull request #1501 from hashicorp/f-stats-opt-in
Allow operators to opt into publishing node and alloc metrics
2016-08-04 13:33:56 -07:00
Diptanu Choudhury 531b619ce4 Merge pull request #1475 from mwieczorek/windows-hostIp-portBindings
Empty host ip for windows containers port bindings
2016-08-04 13:30:43 -07:00
Alex Dadgar 1fe4158097 Merge pull request #1519 from vrenjith/master
Remove docker volumes while removing container
2016-08-04 12:54:00 -07:00