Commit Graph

1939 Commits

Author SHA1 Message Date
Michael Schurter 307e674eca Flip disco chan; clarify method names/comments 2016-09-26 15:52:40 -07:00
Michael Schurter 888ee21270 Return csv of servers from Stats, not just count 2016-09-26 15:40:26 -07:00
Alex Dadgar b28e817b1d Test fix 2016-09-26 15:35:59 -07:00
Michael Schurter 7dc0079dd2 doDisco -> triggerDiscoveryCh; discovered -> serversDiscoveredCh
Also fix log line formatting
2016-09-26 15:21:28 -07:00
Michael Schurter 434e4be97c noServers -> noServersErr 2016-09-26 15:12:35 -07:00
Michael Schurter b2ddb85a78 consul -> Consul 2016-09-26 15:06:57 -07:00
Diptanu Choudhury 12c7873db2 Closing files when files are removed 2016-09-23 22:17:53 -07:00
Michael Schurter 37cfb2769c Replace periodic handlers with event driven disco
Remove use of periodic consul handlers in the client and just use
goroutines. Consul Discovery is now triggered with a chan instead of
using a timer and deadline to trigger.

Once discovery is complete a chan is ticked so all goroutines waiting
for servers will run.

Should speed up bootstraping and recovery while decreasing spinning on
timers.
2016-09-23 17:02:48 -07:00
Michael Schurter 2ab5264595 Retry all servers on RPC call failure
rpcproxy is refactored into serverlist which prioritizes good servers
over servers in a remote DC or who have had a failure.

Registration, heartbeating, and alloc status updating will retry faster
when new servers are discovered.

Consul discovery will be retried more quickly when no servers are
available (eg on startup or an outage).
2016-09-23 11:44:48 -07:00
Diptanu Choudhury 589356fd55 Adding a snapshot endpoint on the client (#1730) 2016-09-21 21:28:12 -07:00
Alex Dadgar 12de69a66f Struct and parse 2016-09-21 11:31:09 -07:00
Alex Dadgar 50efdb00e9 Merge pull request #1713 from hashicorp/f-alloc-runner-vault
Vault integration in client
2016-09-20 16:15:55 -07:00
Alex Dadgar 64de46432a Merge pull request #1677 from hashicorp/f-vault-implicit-constraint
Vault implicit Task Group constraint + allow root tokens
2016-09-20 16:15:32 -07:00
Diptanu Choudhury f7a9b39e8c Ensuring that we are not emitting stats when handle is nil (#1723)
* Ensuring that we are not emitting stats when handle is nil

* Updated the changelog
2016-09-20 11:29:34 -07:00
Alex Dadgar 83905075e5 Fix comment 2016-09-17 11:31:17 -07:00
Alex Dadgar 40fc1d1dfd Task runner test 2016-09-15 17:39:08 -07:00
Alex Dadgar 0fefaef008 Alloc runner tests 2016-09-15 17:24:09 -07:00
Alex Dadgar 0f40bd41a3 Handle recovery failure 2016-09-15 12:50:44 -07:00
Alex Dadgar 688e616200 Fix token renewal 2016-09-15 11:20:51 -07:00
Alex Dadgar ec152a6d12 Clean up vault client 2016-09-14 18:10:56 -07:00
Alex Dadgar 6702a29071 Vault token threaded 2016-09-14 13:30:01 -07:00
Robert Neumayer 8dc19dbd10 Log adding of servers at INFO level 2016-09-14 22:24:17 +02:00
Michael Schurter 9a2a17b48f Merge pull request #1682 from hashicorp/b-sanity-check-state-file
Prevent state file corruption and check state file sanity on save
2016-09-13 11:24:34 -07:00
Michael Schurter cd8606b9e3 Revert "A nil context isn't an error"
This reverts commit fe9fe4c26259c1ad3bd7e94bd711418aaf819b20.
2016-09-12 12:56:12 -07:00
Diptanu Choudhury ab6a9a5120 Fixing alloc runner tests 2016-09-04 19:09:08 -07:00
Diptanu Choudhury 0f77d81f7d Merge pull request #1683 from mwieczorek/enable-syslog-for-windows
Enable syslog for windows
2016-09-04 17:07:25 -07:00
Michal Wieczorek 94071fc294 Enable syslog server and universal collector for windows 2016-09-05 00:26:36 +02:00
Michael Schurter 8a57913a44 A nil context isn't an error 2016-09-02 16:24:53 -07:00
Michael Schurter f601361d58 Don't serialize task states twice in state files 2016-09-02 16:07:06 -07:00
Michael Schurter 6cb6d9cdf1 Lock around saving state
Prevent interleaving state syncs as it could conceivably lead to
empty state files as per #1367
2016-09-02 16:07:06 -07:00
Michael Schurter e7dd443447 Add sanity check to SaveState
Also just reuse the task states snapshot taken by `Alloc()` instead of
doing a redundant copy.
2016-09-02 16:07:06 -07:00
Alex Dadgar eecef73302 syscall error 2016-09-02 15:00:46 -07:00
Alex Dadgar eef786dd9d Secret dir materialized in alloc/task directory 2016-09-02 12:44:05 -07:00
Alex Dadgar 2c8dd8bbd3 Revert "Introduce a Secret/ directory" 2016-09-01 17:23:15 -07:00
Alex Dadgar 4a8fba5cf7 small fixes 2016-09-01 13:38:31 -07:00
Alex Dadgar b0adaa5301 Allow root token 2016-09-01 12:05:08 -07:00
Alex Dadgar 8ca3a16825 Fingerprint 2016-09-01 11:10:14 -07:00
Alex Dadgar 1ed454dd60 Merge pull request #1671 from hashicorp/f-secret-dir2
Introduce a Secret/ directory
2016-09-01 09:56:17 -07:00
Alex Dadgar 9fa23e3536 Symlink on windows 2016-08-31 21:41:44 -07:00
Alex Dadgar 5d3b47e648 Address comments and reserve 2016-08-31 18:11:02 -07:00
Michael Schurter cbb0b8e31e Merge pull request #1668 from hashicorp/b-fix-consul-updates
Fix old services not getting removed from consul on update
2016-08-31 17:17:09 -07:00
Alex Dadgar 0626eb9619 environment variables 2016-08-31 13:56:11 -07:00
Alex Dadgar d59e14eed4 Interface + tests 2016-08-30 21:40:32 -07:00
Vishal Nayak b6b73545ea Merge pull request #1606 from hashicorp/f-vault-client
VaultClient for Nomad client's interactions with Vault
2016-08-30 13:13:54 -04:00
vishalnayak d0ad1603c3 Print debug message only when error is non-nil 2016-08-30 13:14:34 -04:00
vishalnayak 55a6f06e15 Addressed review feedback 2016-08-30 13:08:13 -04:00
vishalnayak 3808dd0ff8 Return only fatal error to renewal error channel 2016-08-30 12:46:59 -04:00
vishalnayak a0dbfe25b3 Fix tests 2016-08-29 21:30:06 -04:00
vishalnayak 82f6209e97 tokenDeriver function pointer to derive tokens.
Remove rpc*, connPool, node and region from vaultclient.
2016-08-29 20:32:05 -04:00
Alex Dadgar 14b7126511 Secret dir, hello world 2016-08-29 15:41:52 -07:00
vishalnayak f35bb409b6 Use Job.LookupTaskGroup 2016-08-29 16:34:39 -04:00
vishalnayak 160ba48eb4 Address review feedback 2016-08-29 12:47:33 -04:00
Alex Dadgar aaca0bdaf4 Make maxSize exported so that it is serialized 2016-08-28 17:48:35 -07:00
Michael Schurter 32626ac608 Remove unused embedded lock on ExecContext 2016-08-26 11:58:03 -07:00
Michael Schurter d31f373a5b Merge pull request #1653 from hashicorp/b-fix-artifact-retry
Don't fail other tasks when retrying artifact get
2016-08-26 09:53:39 -07:00
Michael Schurter 592b898f43 Make sure bad doesn't fail before web runs 2016-08-25 17:25:51 -07:00
Michael Schurter 139d2e7939 Ensure bad task reaches terminal state within test 2016-08-25 16:05:19 -07:00
Michael Schurter a739058fa3 Ensure web task exited successfully
Web task should run to completion successfully while the `bad` task is
retrying artifact downloads.
2016-08-25 14:42:50 -07:00
Michael Schurter 5ce26f82fe Don't fail other tasks when retrying artifact get
The artifact fetching may be retried and succeed, so don't set the task
as dead.

Fixes #1558
2016-08-25 13:16:41 -07:00
Ivo Verberk d89dc40fe2 Small comment fix. 2016-08-25 20:50:11 +02:00
Ivo Verberk 57012e8d8c Monitor the complete alloc directory, not just the shared part. 2016-08-25 20:48:19 +02:00
Ivo Verberk 9113244131 Don't duplicate TaskKilled event and check for TaskSiblingFailed. 2016-08-25 20:11:10 +02:00
Diptanu Choudhury e9b27a528c Fixed the raw_exec fingerprint test 2016-08-24 13:38:43 -05:00
Diptanu Choudhury 10e10b2fe1 fixed the qemu fingerprinter test 2016-08-24 12:25:02 -05:00
vishalnayak 56e42cf03d Employ DeriveVaultToken API and flesh-up DeriveToken 2016-08-24 12:29:59 -04:00
vishalnayak 6002e596c4 VaultClient for Nomad Client 2016-08-24 09:43:45 -04:00
Diptanu Choudhury 9309247f9d Fixed a java test 2016-08-23 16:51:09 -05:00
Diptanu Choudhury 05fe72e89e fixed the exec fingerprinter test 2016-08-23 16:40:56 -05:00
Diptanu Choudhury 588a4802c1 removing driver_mock 2016-08-23 16:05:03 -05:00
Alex Dadgar 0f3ec9c759 Fix TestDockerDriver_Fingerprint 2016-08-23 10:39:40 -07:00
Alex Dadgar 1da8566322 Merge pull request #1580 from hashicorp/f-disk-usage-monitoring
Monitor and enforce shared allocation directory disk usage
2016-08-23 09:49:53 -07:00
Diptanu Choudhury 1e1eef56a1 Putting the mock driver behind a build flag 2016-08-22 15:02:28 -05:00
Diptanu Choudhury 4ca623bcfe blocking chained allocations until previous allocation hasn't terminated 2016-08-22 11:34:24 -05:00
Kenjiro Nakayama b06c6d9311 driver.docker: tiny: debug messages output task name instead of image name 2016-08-21 19:51:32 +09:00
Alex Dadgar a90dafe9ab handle the upgrade case 2016-08-18 19:01:24 -07:00
Ivo Verberk 2a17895a83 Disk resource monitoring and enforcement 2016-08-18 07:59:03 +02:00
Alex Dadgar 895c31f605 Nodes generate Secret ID and used for retrieving allocations and registering 2016-08-17 16:31:47 -07:00
vishalnayak ff9bb5b08b Disable Vault instead of supplying test token 2016-08-17 16:25:38 -07:00
vishalnayak 1c1457b01b Fix a few client tests 2016-08-17 16:25:38 -07:00
Alex Dadgar 7d899b6c60 Pass Vault config to client 2016-08-17 16:23:29 -07:00
Diptanu Choudhury dc0e395982 re-using copyimage 2016-08-17 15:25:03 -07:00
Diptanu Choudhury 0d7cd53c63 Fixed docker tests 2016-08-17 15:25:03 -07:00
Diptanu Choudhury ab7f8847c1 changing error statement 2016-08-17 13:48:31 -07:00
Diptanu Choudhury 2968ce9399 Fixed the docker script check 2016-08-17 13:23:48 -07:00
Alex Dadgar 3b9188fcf0 Merge pull request #1598 from nak3/rkt-fix5
driver.rkt: Remove unnecessary job validation
2016-08-16 15:12:45 -07:00
Alex Dadgar 423da3e99d Merge pull request #1593 from hashicorp/f-rereg
Reregister Client on failed heartbeat
2016-08-16 13:29:34 -07:00
Kenjiro Nakayama c97beb8deb driver.rkt: Remove unnecessary job validation 2016-08-16 23:33:34 +09:00
ramukima 4f18963a97 go fmt performed code when copied from another directory got messed up again ? Ok, ran go fmt again 2016-08-16 09:20:13 -04:00
ramukima 17b902f20f issue-1588 : Allow extra driver config args as a passthrough for qemu executable from a task specification 2016-08-15 23:36:13 -04:00
Alex Dadgar 84820db86f If the client detects that a heartbeat has failed because it is not registered, reregister 2016-08-15 17:24:09 -07:00
Alex Dadgar 64b86bed0d Merge pull request #1581 from nak3/fix-rkt2
Set host environment variables to taskEnv of rkt driver
2016-08-15 10:31:10 -07:00
Alex Dadgar a2b603bd82 Merge pull request #1586 from nak3/rkt-fix3
tiny: Catch error returned from SyncServices in rkt driver
2016-08-15 10:29:28 -07:00
Kenjiro Nakayama d0ccdacd08 rkt.driver: Fix wrong MB calculation 2016-08-14 14:27:42 +09:00
Kenjiro Nakayama 906526fbe0 tiny: Catch error returned from SyncServices in rkt driver 2016-08-14 13:11:17 +09:00
Kenjiro Nakayama 0efc0e0231 Set host environment variables to taskEnv of rkt driver 2016-08-14 00:42:53 +09:00
Diptanu Choudhury 8c8d00dd5b Merge pull request #1572 from hashicorp/fix-docker-test
Fixed docker tests
2016-08-12 13:04:33 -07:00
Diptanu Choudhury f3420c65e3 Updated the busybox images 2016-08-12 11:39:58 -07:00
Diptanu Choudhury dd7e69006e Not running tests parallal 2016-08-11 21:53:27 -07:00
Alex Dadgar b34d65f1a5 Update test.sh 2016-08-11 21:35:50 -07:00
Diptanu Choudhury 839ecd1df6 Fixed docker tests 2016-08-11 19:28:41 -07:00
Alex Dadgar ed23dff23c Merge pull request #1571 from hashicorp/t-logs-tests
Fixing logs tests on travis
2016-08-11 19:24:46 -07:00
Alex Dadgar b3dca54a1a Wrap file rotator tests in wait for 2016-08-11 19:23:03 -07:00
Alex Dadgar 8323b6a0b5 only use polling 2016-08-11 18:59:48 -07:00
Diptanu Choudhury b71f687f62 Merge pull request #1413 from hashicorp/b-tests
Fix flaky tests
2016-08-11 18:15:29 -07:00
Alex Dadgar 2614b0f06e Update task_runner_test.go 2016-08-11 18:01:27 -07:00
Alex Dadgar ac7749f70e Fix task runner test 2016-08-11 13:16:17 -07:00
Kenjiro Nakayama fe13453012 Update after the review 2016-08-11 10:53:33 +09:00
Kenjiro Nakayama c3b871e90d Return error when client failed to collect host stats 2016-08-11 09:38:28 +09:00
Diptanu Choudhury 9a75052d2c Merge pull request #1518 from pubnub/feature/chroot-map-rebase
Add config field to specify chroot mapping for exec driver
2016-08-10 17:00:03 -07:00
Diptanu Choudhury 28b3f511e0 Fixed some error messages 2016-08-10 15:17:32 -07:00
Diptanu Choudhury 1b74f863fe Merge pull request #1533 from nak3/fix-error-in-client
tiny: Return fmt.Errorf instead of duplicated error messages
2016-08-10 15:13:47 -07:00
Kenjiro Nakayama 6ec6c27cb4 Update debug option from string to bool 2016-08-09 16:51:00 +09:00
Kenjiro Nakayama 11a8a7218e Add debug option to rkt task config 2016-08-09 09:01:05 +09:00
Kenjiro Nakayama 6a810e6f1e Update after review 2016-08-09 08:57:26 +09:00
Kenjiro Nakayama 5c621b74e5 tiny: Return fmt.Errorf instead of duplicated error messages 2016-08-09 08:57:26 +09:00
Jay Oster 09113ffbc8 Fix Linux executor isolation test
- Properly expects the hard-coded mounts (alloc, dev, and proc) and hardcoded local directories (local and tmp)
- Also verifies that etc contains only the requested paths
2016-08-08 14:04:09 -07:00
Diptanu Choudhury c0ec1b2101 Merge pull request #1532 from nak3/fix-fingerprint-cpu-log
tiny: Fix duplicated error message in CPU fingerprint
2016-08-08 13:44:49 -04:00
Diptanu Choudhury 70d2f8ef1d Merge pull request #1534 from nak3/fix-intask_runner
tiny: print task name and error message for SaveState error
2016-08-08 13:37:25 -04:00
Kenjiro Nakayama e7863ea8ee tiny: print task name and error message for the SaveState error in task_runner 2016-08-07 13:33:58 +09:00
Kenjiro Nakayama 71371fc592 tiny: Fix duplicated error message in CPU fingerprint 2016-08-07 12:49:40 +09:00
Kenjiro Nakayama 60b58eed84 Update GetArtifact by removing unused logger 2016-08-06 23:37:32 +09:00
Diptanu Choudhury fb178a1f5f Merge pull request #1477 from nak3/add-syslog_server-test
Add Syslog server start shutdown test
2016-08-05 12:01:24 -07:00
Alex Dadgar ddd8adce96 changelog + use driver config 2016-08-05 10:55:20 -07:00
Alex Dadgar f8e4bc73c9 Merge pull request #1493 from nak3/rkt-fix1
Pass command and trust_prefix to the validation of rkt task configuration
2016-08-05 10:52:02 -07:00
Alex Dadgar 096956257d changelog 2016-08-05 10:47:44 -07:00
Kenjiro Nakayama fc1195b7ea Add Syslog server start shutdown test 2016-08-06 02:01:33 +09:00
Michal Wieczorek b688261a99 Set windows containers default network mode to 'nat' 2016-08-05 06:01:26 +02:00
Jay Oster 2ae059b41d Address review comments
- Simplify map length check in Linux Executor
- Added a `chroot_env` test for config parser
- Moved `ChrootEnv` field from ExecutorCommand to ExecutorContext
- Added a test for `chroot_env` functionality
2016-08-04 15:33:06 -07:00
Diptanu Choudhury 5ff750db96 Merge pull request #1501 from hashicorp/f-stats-opt-in
Allow operators to opt into publishing node and alloc metrics
2016-08-04 13:33:56 -07:00
Diptanu Choudhury 531b619ce4 Merge pull request #1475 from mwieczorek/windows-hostIp-portBindings
Empty host ip for windows containers port bindings
2016-08-04 13:30:43 -07:00
Alex Dadgar 1fe4158097 Merge pull request #1519 from vrenjith/master
Remove docker volumes while removing container
2016-08-04 12:54:00 -07:00
Jay Oster 24e8f752ab Add chroot_env to Java driver (which uses the exec driver internally) 2016-08-04 11:15:35 -07:00
Kenjiro Nakayama cd12645e4c Add TestRktTaskValidate 2016-08-04 23:15:13 +09:00
Kenjiro Nakayama bf27963903 Add TestRktTrustPrefix 2016-08-04 17:26:10 +09:00
Kenjiro Nakayama 1176fd123e Pass command and trust_prefix to the validation of rkt task configuration 2016-08-04 17:24:56 +09:00
vrenjith 41cf7cc623 Update docker.go
Remove container volumes
2016-08-04 11:43:50 +05:30
vrenjith 4e603e1306 Update checks_test.go
Remove docker volumes while exiting container
2016-08-04 11:42:47 +05:30
Jay Oster 7df692226a Add config field to specify chroot mapping for exec driver
- Same format as used by the internal chroot mapping
- Map: source_path -> dest_path
- Example HCL:

client {
  chroot_env {
    "/etc" = "/etc"
    "/lib" = "/lib"
    "/opt/projects/foo/bin" = "/usr/bin"
  }
}
2016-08-03 17:17:17 -07:00
Mathias Lafeldt 0727db4ca0
Test configuration of Docker working directory 2016-08-03 16:35:49 +02:00
Mathias Lafeldt d91f7dbdf8
Docker driver: allow to configure working directory 2016-08-03 16:18:15 +02:00
Alex Dadgar 47f5c8f523 use priviledge of the config 2016-08-02 16:10:15 -07:00
Alex Dadgar cec6d8a1eb remove gating of ipc, user ns and pidmode based on hosts priviledge mode config 2016-08-02 16:02:34 -07:00
Mathias Lafeldt acbee08a0a
Fix typo: atttempts 2016-08-02 18:11:03 +02:00
Diptanu Choudhury 41b540fbc8 Allow operators to opt into publishing node and alloc metrics 2016-08-01 19:52:20 -07:00
Kenjiro Nakayama e8ce8408a4 Fix gofmt in restarts_test.go 2016-07-30 21:11:06 +09:00
Cameron Davison 777bdf4a1e
fix setup consul syncer error message 2016-07-28 22:14:52 -05:00
Alex Dadgar 2999c12ef1 disable swap 2016-07-28 12:17:00 -07:00
Michal Wieczorek 4b82b6c3d4 Empty host ip for windows containers port bindings 2016-07-28 00:00:57 +02:00
Diptanu Choudhury 50842b88c7 Fixed some bugs 2016-07-25 17:26:38 -07:00
Alex Dadgar 42df093939 Merge pull request #1456 from hashicorp/b-system-job
Node Register handles transistioning to ready and creating evals
2016-07-25 12:46:35 -07:00
Alex Dadgar 3ea95bb91c initial log api impl 2016-07-25 11:16:01 -07:00
Alex Dadgar 84c3711989 Merge pull request #1457 from hashicorp/f-kill-event
Add killing event and mark task as not running when killed
2016-07-22 17:33:18 -07:00
Alex Dadgar 3ec9cf3e0d Merge pull request #1454 from hashicorp/b-blocking-stats
Driver Kill() does not block Stats()
2016-07-22 17:29:52 -07:00
Michal Wieczorek b6b3e24541 Link speed for windows network fingerprinting - tests 2016-07-22 22:49:03 +02:00
Alex Dadgar 90748cedad Add killing event and mark task as not running when killed 2016-07-21 15:49:54 -07:00
Alex Dadgar ebac5cb283 Node.Register handles the case of transistioning to ready and creating evals 2016-07-21 15:22:02 -07:00
Alex Dadgar 898435d372 Retrieve task runners in helper 2016-07-21 13:41:01 -07:00
Michal Wieczorek 679fefc155 Link speed for windows network fingerprinting 2016-07-20 22:13:50 +02:00
Diptanu Choudhury 22af229cef Merge pull request #1321 from mwieczorek/f-windows-binds
Volume binds for windows containers
2016-07-18 10:20:44 -06:00
Alex Dadgar c8e7b909c7 Merge pull request #1404 from hashicorp/f-streaming
Implement a streaming API and tail in the fs command
2016-07-12 17:23:04 -06:00
Alex Dadgar 661d100f2f address comments 2016-07-12 17:01:33 -06:00
Alex Dadgar 807bf3cf6c tests wait for the container to start 2016-07-12 11:36:06 -06:00
Alex Dadgar caa0e48841 Debug timeout 2016-07-12 11:07:05 -06:00
Alex Dadgar e1d68f3e9b Docker host network test 2016-07-12 10:59:34 -06:00
Alex Dadgar 9085b5ca0d Merge pull request #1405 from novilabs/delay-on-startup-failure
do not fail for multiple startup failures, delay instead
2016-07-12 09:51:40 -06:00
Cameron Davison d7fe56ecf3 test policy delay for startup error 2016-07-11 20:54:36 -05:00
Cameron Davison dd314ea06e if policy mode is delay, do not fail for multiple startup failures, delay instead 2016-07-11 20:40:53 -05:00
Diptanu Choudhury ed3be34105 Introduced an env var for rkt tests 2016-07-11 15:48:16 -06:00
Diptanu Choudhury 4e0b621ffa Skipping travis tests and not installing rkt on travis 2016-07-11 15:10:09 -06:00
Sean Chittenden a9b3f5e552
Alpha-sort the build platforms 2016-07-11 12:23:46 -07:00
Sean Chittenden 267198742f
Merge branch 'master' into f-resource-isolation-fingerprinter 2016-07-11 12:23:09 -07:00
Sean Chittenden d309649ada
Darwin currently has allocdir support.
Pointed out by: @dadgar
2016-07-11 12:19:17 -07:00
Sean Chittenden 20d87f1782
Remove cgroup fingerprinter from non-linux systems.
If someone wants to extend or reuse Cgroup detenction in the future they
can move `cgroup_linux.go` to `cgroup.go` and add the relevant build
tags.

Requested by: @dadgar
2016-07-11 12:16:56 -07:00
Diptanu Choudhury e2909db9ef Merge pull request #1388 from novilabs/support-docker-syslog-unixformat-and-defaultformat
Support docker syslog unixformat and defaultformat
2016-07-11 11:17:30 -07:00
Alex Dadgar f11b1ce079 Get windows to build 2016-07-11 11:52:41 -06:00
Alex Dadgar e9ffadfdc6 initial comments 2016-07-11 10:58:18 -06:00
Sean Chittenden 9966169596
Merge branch 'f-resource-isolation-cleanup' into f-resource-isolation-fingerprinter 2016-07-11 00:10:21 -07:00
Sean Chittenden d20c5fc327 Merge pull request #1402 from hashicorp/f-resource-isolation-cleanup
Resource isolation cleanup
2016-07-11 02:09:35 -05:00
Sean Chittenden be272168c7
Rename resourceContainer{,Context} and resCon{,Ctx}. 2016-07-11 00:02:55 -07:00
Sean Chittenden 1c14e01ac0
Add a comment describing IsolationConfig 2016-07-10 23:45:44 -07:00
Sean Chittenden 5e2e5b6ccc
Merge branch 'b-exec-cleanup' into f-resource-isolation-cleanup 2016-07-10 23:41:04 -07:00
Sean Chittenden cf6b97ec6c Merge pull request #1400 from hashicorp/b-exec-cleanup
Initialize the list of available fingerprinters per platform.
2016-07-10 23:22:02 -07:00
Sean Chittenden f39e84b672
Improve readability: use of a switch vs two if's 2016-07-10 20:18:57 -07:00
Sean Chittenden 2ffbeee06c
Skip the network fingerprinter test when offline.
Conditionalize the network fingerprinter test so that it works when a
user is offline.  Similarly, when the network fingerprint test fails in
the future pass a HINT to the user to set an env var to allow the test
to be skipped in the future.
2016-07-10 20:16:06 -07:00
Sean Chittenden 2983bd6fce
Fix test for non-Linux platforms.
The following tests now check a whitelist for whether or not their
driver is present or not, or if the OS is supported or not.

* `TestAllocDir_MountSharedAlloc`
* `TestClient_Drivers_InWhitelist` (`exec` driver)
* `TestClient_Drivers` (`exec` driver)
* `TestJavaDriver_Fingerprint` (`java` driver)
2016-07-10 15:19:49 -07:00
Sean Chittenden 710173e9cb
Build the Cgroup fingerprinter on only Linux.
Change the logic from `!linux` to an empty build tag so that *if*
another platform picks up Cgroups support they can add themselves to the
necessary build tags for this fingerprinter and be on their way.
Because this technology isn't inherently Linux-specific and isn't
mutually exclusive of other resource isolation containers, resist the
urge to rename the Cgroup fingerprinter to something generic like the
ResourceContainerFingerprinter.
2016-07-10 13:55:06 -07:00
Alex Dadgar 51ae7ace25 initial tail impl 2016-07-10 13:57:04 -04:00
Sean Chittenden d4fe69ddf9
Update comments and pushdown a lock into the resource container 2016-07-10 00:12:59 -07:00
Sean Chittenden fc9cd8d4af
Push down the Linux-specific bits into resourceContainer 2016-07-10 00:06:53 -07:00
Sean Chittenden 6fc269d2a6
Move unit tests around into per-platform where appropriate. 2016-07-09 23:56:31 -07:00
Sean Chittenden a5dc6c2da9
Push the Client's cleanup of Cgroups down 2016-07-09 23:45:33 -07:00
Sean Chittenden 5dbc0bf382
Rename resourceContainer.cleanup() to executorCleanup()
Not to be confused with the imminent ClientCleanup().
2016-07-09 23:25:33 -07:00
Sean Chittenden 5f8f0a50ac
Begin cgroup pushdown into platform specific files 2016-07-09 23:01:14 -07:00
Sean Chittenden bdd7022fdc
Centralize the fingerprintrs.
Add platform specific fingerprinters per platform.

Requested by: @diptanu
2016-07-09 22:31:14 -07:00
Sean Chittenden 1e2e0ca050
Initialize the list of available fingerprinters per platform. 2016-07-09 00:22:42 -07:00
Diptanu Choudhury 5b39a5db40 Fixed a debug message 2016-07-09 00:12:53 -07:00
Diptanu Choudhury e5310b76a6 Merge pull request #1399 from hashicorp/b-exec-cleanup
WIP: Cleanup exec driver
2016-07-09 00:08:43 -07:00
Sean Chittenden 03c571c61b
Consolidate fingerprinters into a single `map`. 2016-07-08 23:37:14 -07:00
Sean Chittenden 7530b27014 Move all non-Linux Fingerprinter items to the default exec driver 2016-07-08 18:35:46 -07:00
Diptanu Choudhury 5d61fa01f1 Fixed tests 2016-07-08 18:27:51 -07:00
Diptanu Choudhury 3c4002c48b Fixed the client tests 2016-07-08 17:49:58 -07:00
Diptanu Choudhury 1784d98182 Fixed the host port environment variable 2016-07-08 15:37:44 -07:00
Cameron Davison 921a6c889c remove the expected leading space, after the colon in syslog 2016-07-06 11:08:24 -05:00
Cameron Davison 07a9e15560 get into the hour minute second part of the time before looking for spaces, and then looking for the : seperator 2016-07-06 11:08:24 -05:00
Alex Dadgar c35b1be845 Set running when restoring 2016-06-28 13:47:59 -07:00
Wojciech Bederski a73422b4ff Fix docker driver lockup during nomad boot
Unit mismatch caused docker driver to wait almost indefinitely during boot 
(when one or more containers were a bit uncooperative during StopContainer())
This should fix problems described in  #1202
2016-06-28 14:26:47 +02:00
Cameron Davison d1e7d9c50f
write state to temp file and then rename 2016-06-27 12:29:33 -05:00
Jake Champlin f094969c7b
Update failing tests 2016-06-23 11:28:17 -04:00
Alex Dadgar 14e950f882 Treat float as int 2016-06-22 15:09:39 -07:00
Alex Dadgar 4ff8edd2da Floor CPU MHz and total compute and mark hostname as unique 2016-06-22 15:01:36 -07:00
Diptanu Choudhury 0a10873aa6 Merge pull request #1335 from hashicorp/f-set-docker-timeout
Setting a timeout in the docker client
2016-06-21 17:00:14 -07:00
Diptanu Choudhury 2837d3395d Setting a timeout in the docker client 2016-06-21 16:58:21 -07:00
Diptanu Choudhury 1d5c5b18f3 Making SSL default 2016-06-21 16:41:14 -07:00
Sean Chittenden 8bdb38d016
Code golf
Pointed out by: @dadgar
2016-06-21 14:26:01 -07:00
Sean Chittenden df4fe2e502
Fix the shuffling of remote datacenters.
Pointed out by: @ryanuber
2016-06-21 13:37:22 -07:00
Diptanu Choudhury 88ac1b33a4 Not emitting per-pid stats and added the total ticks consumed by a Task 2016-06-20 17:30:25 -07:00
Alex Dadgar 19024f4da0 Merge pull request #1322 from hashicorp/b-docker-logs-splicing
Make line copy to avoid being overriden by subsequent scans
2016-06-20 13:17:49 -07:00
Alex Dadgar 661dc200f3 Make line copy to avoid being overriden by subsequent scans 2016-06-20 13:14:43 -07:00
Michal Wieczorek 67a04bb1cc Volume binds for windows containers 2016-06-20 21:46:33 +02:00
Alex Dadgar 3cd9c9590b guard against NaN 2016-06-20 10:29:46 -07:00
Alex Dadgar 7b83503596 finer grain locking 2016-06-20 10:19:06 -07:00
Alex Dadgar 744270590b Guard against bad restore 2016-06-17 14:58:53 -07:00
Alex Dadgar c9f7467ccb Driver tests use client default config 2016-06-17 14:24:49 -07:00
Sean Chittenden 7b9961f09b
Initialize the stats helpers before accessing them for the first time 2016-06-17 13:23:30 -07:00
Sean Chittenden 9cb649d247 Merge pull request #1307 from hashicorp/f-fingerprint-cpus
F fingerprint cpus
2016-06-17 12:23:40 -07:00
Sean Chittenden 9e287858de Merge pull request #1310 from hashicorp/b-logger
Create and pass only one `logger` object around per Agent
2016-06-17 12:16:35 -07:00
Sean Chittenden 21b84fc3e6
Memoize the CPU stats. Error if CPU fingerprinting fails. 2016-06-17 12:13:53 -07:00
Alex Dadgar 27c6398639 debug message when stopping container 2016-06-17 11:52:44 -07:00
Sean Chittenden 686c125fea
Record and use only the first Mhz from the CPU fingerprinter.
Assume all cores are the same speed.
2016-06-17 11:06:57 -07:00
Sean Chittenden 46e2d54acf
Provide `nomad.Config` with a default `LogOutput` of `os.StdErr` 2016-06-17 06:44:10 -07:00
Sean Chittenden 9a60999100
Pass a logger arg to `NewClient` and `NewServer` 2016-06-16 23:29:23 -07:00
Sean Chittenden 4cc90753f8
In the debug log, split the unit from the measurement
awk(1) friendly is UNIX(tm) friendly.
2016-06-16 23:07:13 -07:00
Sean Chittenden 2dcb591cd8
Warn when we're unable to fingerprint the CPU Mhz 2016-06-16 23:07:13 -07:00
Sean Chittenden b8e63411c0
Explicitly call `cpu.Counts()` to determine the CPU core count
Much safer than counting the number of InfoStat structs returned.
2016-06-16 23:07:13 -07:00
Sean Chittenden d17af396ca
Create config.DefaultConsulConfig() 2016-06-16 20:41:05 -07:00
Sean Chittenden fd18eb7fdb
Only register the Client services reaper when `consul.auto_advertise` is enabled 2016-06-16 18:24:58 -07:00
Sean Chittenden 1ce2cc6141
`conf` -> `config` 2016-06-16 17:05:29 -07:00
Sean Chittenden 015248a67f
Fix tests for NewServer() in client mode.
Pointy-hat: sean-

I'm positive I've done this before.
2016-06-16 16:34:22 -07:00
Sean Chittenden 952b6ce7b5
Only auto-join clients if `client_auto_join` is true 2016-06-16 14:47:21 -07:00
Sean Chittenden ec77a1869e
Test for errors 2016-06-16 14:43:46 -07:00
Sean Chittenden af55b74114 Merge pull request #1276 from hashicorp/f-consul-server-autojoin
Teach Nomad servers how to fall back to Consul.
2016-06-16 14:40:45 -07:00
Diptanu Choudhury ed67f1a347 Merge pull request #1285 from hashicorp/fix-selinux-options
Added a client options for setting selinux options
2016-06-16 22:45:24 +02:00
Diptanu Choudhury 266c417ac8 Changed the client options for docker volume selinux labels 2016-06-16 21:41:02 +01:00
Alex Dadgar fe588a2469 Guard against restoring a nil task in task_runner 2016-06-16 11:55:40 -07:00
Sean Chittenden 008d75184b
Use the `%+q` verb in log messages (vs `%q`). 2016-06-16 11:03:51 -07:00
Alex Dadgar 7375d828e1 remove trace 2016-06-15 15:47:59 -07:00
Sean Chittenden 5e0ced2ae7
Shuffle all datacenters vs only the nearest N datacenters.
Per discussion, we want to be aggressive about fanning out vs possibly
fixating on only local DCs.  With RPC forwarding in place, a random walk
may be less optimal from a network latency perspective, but it is guaranteed
to eventually result in a converged state because all DCs are candidates
during the bootstrapping process.
2016-06-15 12:40:51 -07:00
Sean Chittenden 2123460cf0
Bump various Consul search limits
Client: Search limit increased from 4 random DCs to 8 random DCs, plus nearest.
Server: Search factor increased from 3 to 5 times the bootstrap_expect.

This should allow for faster convergence in large environments (e.g.
sub-5min for 10K Consul DCs).
2016-06-15 12:40:51 -07:00
Alex Dadgar cf99fc3173 Use Status.Peers instead of Status.Ping 2016-06-15 12:00:20 -07:00
Diptanu Choudhury fa216199ce Added documentation 2016-06-15 02:42:15 +02:00