luxolus/open-nomad
2
0
Fork 0
Code Issues 1 Pull requests Packages Releases Wiki Activity
11031 commits 1 branch 0 tags 311 MiB
Commit graph

2940 commits

Author SHA1 Message Date
Alex Dadgar 540007c1a3 Merge pull request #237 from hashicorp/f-raw-exec-driver 
A raw fork/exec driver that provides no isolation.
 2015-10-09 11:30:12 -07:00
Alex Dadgar db0ba9688d Better parsing of raw_exec option and updated docs 2015-10-09 11:29:59 -07:00
Daniel Imfeld cbe349b8b4 Add a few more GCE-specific attributes: 
* cpu-platform
* scheduling.automatic-restart
* scheduling.on-host-maintenance
* network.NETWORKNAME=true
 2015-10-09 09:10:40 -05:00
Chris Bednarski 0d6c75eefb Merge pull request #225 from hashicorp/b-docker-port 
Change docker default port to support windows and boot2docker
 2015-10-08 23:50:11 -07:00
Alex Dadgar 1570cf1a77 Merge pull request #240 from crosbymichael/wait 
Reap process after sending kill signal
 2015-10-08 17:10:23 -07:00
Michael Crosby 5fee9f37ec Reap process after sending kill signal 
Without waiting on the process after sending a kill will cause zombies
and we all know what happens when we have a zombies outbreak.

There are other calls to kill in this file but they are done on the main
process for the task so they should have the wait method called at
sometime in their lifecycle.
 2015-10-08 16:54:41 -07:00
Alex Dadgar cd44cc9ec6 Fix go vet format errors in exec_linux 2015-10-08 16:47:29 -07:00
Alex Dadgar 8130510a5a Fix AWS Networking Fingerprint tests failing when actually on AWS 2015-10-08 12:49:43 -07:00
Chris Bednarski a9f1f64199 Use DefaultDockerHost from fsouza upstream 2015-10-08 12:35:19 -07:00
Alex Dadgar 751a17ef0c Actually add the files 2015-10-08 12:19:39 -07:00
Alex Dadgar 9d1d179deb Change name from pexec to raw_exec; hamming distance one seemed like a bad idea 2015-10-08 12:19:39 -07:00
Alex Dadgar dbbaf03b98 Privileged exec driver 2015-10-08 12:19:39 -07:00
Chris Bednarski be93bbee76 Merge pull request #233 from hashicorp/docs-executor 
Clarified package documentation for executor
 2015-10-08 11:36:11 -07:00
Chris Bednarski 2574655a8c Insert missing 'may be' and clean up some other formatting / phrasing 2015-10-08 01:38:39 -07:00
Chris Bednarski 96a56d865a Tweak phrasing to be clearer *why* that's not what we want 2015-10-08 00:07:07 -07:00
Chris Bednarski e005af18ba Removed wayward 's' 2015-10-08 00:04:45 -07:00
Chris Bednarski ab217e6c51 Clarified package documentation for executor in response to #232 2015-10-08 00:02:46 -07:00
Alex Dadgar 1467cb6131 Merge pull request #165 from achanda/rkt 
Add a driver for rkt
 2015-10-07 16:00:21 -07:00
Abhishek Chanda ba362fae07 Run gofmt 2015-10-07 22:24:16 +00:00
Abhishek Chanda 0e185b6eaf Add an optional exec parameter to task config 
This overrides the default exec command in the ACI
 2015-10-07 15:15:36 -07:00
Daniel Imfeld d66e78c5ca Parse information for all GCE network interface. 
* No longer setting Device name in the network interface since we can't
match up the info here with real device names.
* Add attributes for all external IPs if more than one exists.
 2015-10-07 13:35:20 -05:00
Abhishek Chanda 853cd7fa2d Use Run for adding trust prefix 
The trust needs to be added before anything can progress
 2015-10-07 11:15:17 -07:00
Daniel Imfeld 70350aead9 Compact tags and attribute reading code. 2015-10-07 12:39:23 -05:00
Ivo Verberk c6e1b13b51 Fix vet warnings 2015-10-07 12:26:58 +02:00
Daniel Imfeld dc1c8e5273 TestNetworkFingerprint_notAWS passes even when actually on AWS 2015-10-06 21:21:42 -05:00
Daniel Imfeld 892cad6775 NonXXX tests should pass when actually running in their respective environments. 
Fixes #224
 2015-10-06 21:16:28 -05:00
Chris Bednarski 3c170b70ee Remove panic -- client is nil when there is an error 2015-10-06 19:09:59 -07:00
Chris Bednarski a4c3da6c0c Merge branch 'master' into b-docker-port 2015-10-06 17:54:32 -07:00
Chris Bednarski ec2e3d4f49 Merge pull request #221 from hashicorp/b-docker-tests-osx 
Add skip detection for all docker tests
 2015-10-06 17:53:47 -07:00
Chris Bednarski d046858057 Support boot2docker or VM for dev/test 2015-10-06 17:53:05 -07:00
Chris Bednarski 67e3c4a4f5 Merge branch 'master' into b-docker-port 2015-10-06 16:38:49 -07:00
Alex Dadgar 441b2de4fb Fix alloc_dir error message 2015-10-06 16:38:06 -07:00
Chris Bednarski 88bb01ab6f Fingerprint test should now behave correctly when the executor is not installed; fix some go vet issues 2015-10-06 16:33:02 -07:00
Chris Bednarski 8cd34d3947 Use docker.NewClient; move dockerClient before it's used 2015-10-06 16:26:31 -07:00
Abhishek Chanda bd91c51643 Use Start to run commands 
Run blocks
 2015-10-06 15:56:39 -07:00
Abhishek Chanda ef84902f79 Add prefix to all message logs for rkt driver 2015-10-06 15:56:39 -07:00
Abhishek Chanda 4be849445d Fix function call 
Make it skip if rkt is not installed
 2015-10-06 15:56:39 -07:00
Abhishek Chanda 99b3660047 Register rkt as a built in driver 2015-10-06 15:56:39 -07:00
Abhishek Chanda 528632da3d Add missing import and remove unsued one 2015-10-06 15:56:39 -07:00
Abhishek Chanda 612aa69f34 Do not run containers interactively 2015-10-06 15:56:39 -07:00
Abhishek Chanda 578aa89e81 Do not register to the metadata service 2015-10-06 15:56:39 -07:00
Abhishek Chanda ab6d756dfe Remove a stray comment 2015-10-06 15:56:39 -07:00
Abhishek Chanda 02eb3659c9 Fix constructing the handle 2015-10-06 15:56:39 -07:00
Abhishek Chanda b6b7d9e875 Add a test fort he rkt driver 2015-10-06 15:56:39 -07:00
Abhishek Chanda bcb74acbca Fix name of prefix 2015-10-06 15:56:39 -07:00
Abhishek Chanda 8efe3e18ba Add a driver for rkt 
rkt docs here
https://github.com/coreos/rkt
 2015-10-06 15:56:39 -07:00
Alex Dadgar f40397e6ef Switch to using BlkioWeight 2015-10-06 15:44:01 -07:00
Chris Bednarski 5e73e746dc Merge branch 'docker-use-environment' of https://github.com/carlosdp/nomad into b-docker-port 2015-10-06 15:36:06 -07:00
Chris Bednarski ef3af6996b Updated Throttle Read/Write to match upstream changes in runc; now takes a list of ThrottleDevice structs instead of an int 2015-10-06 15:31:47 -07:00
Chris Bednarski 42afc2bfeb This is needed for 1.4.2 compat; not sure what removed it 2015-10-06 00:32:34 -07:00
Chris Bednarski 58217619e3 Add skip detection for all docker tests 2015-10-06 00:27:05 -07:00
Daniel Imfeld 32d5c61779 Consolidate GCEMetadataClient into EnvGCEFingerprint 
This allows easier reuse of the same client across multiple functions.
 2015-10-05 13:13:25 -05:00
Daniel Imfeld a85dda68fe And add a comment to the constant. 2015-10-05 12:59:02 -05:00
Daniel Imfeld adbb23e267 Use a constant for the default GCE metadata URL 2015-10-05 12:57:45 -05:00
Daniel Imfeld 46bbfc3549 isAWS should return false on GCE 
GCE and AWS both expose metadata servers, and GCE's 404 response
includes the URL in the content, which maatches the regex. So,
check the response code as well and if a 4xx code comes back,
take that to meanit's not AWS.
 2015-10-05 00:42:34 -05:00
Daniel Imfeld a43a262e66 Fingerprinting code for GCE nodes 
This reads the following:

* hostname
* instance id
* machine-type
* zone
* internal IP
* external IP (if any)
* tags
* attributes

Atributes are placed under the platform.gce.attr.* hierarchy.

Tags are set up as platform.gce.tag.TagName=true.
 2015-10-05 00:40:43 -05:00
Alex Dadgar f441c4c65d Cleanup allocations when in dev mode 2015-10-04 13:36:03 -07:00
Carlos Diaz-Padron aaddc7bf6c Use environment to connect to Docker by default 
Uses the environment definition for docker by default. Docker will
default to the unix/tcp socket if the environment is not set.
 2015-10-04 12:59:54 -07:00
Alex Dadgar aeb400cd8a update test to reflect not uppercasing passed env vars 2015-10-02 17:49:18 -07:00
Antoine POPINEAU a03ce2b39c Removed capitalization of user-defined envvars. 2015-10-02 17:46:55 -07:00
Antoine POPINEAU 56944536fc Moved the envvars logic to TaskEnvironment and TaskEnvironmentVariables(). Added tests there. 2015-10-02 17:46:55 -07:00
Antoine POPINEAU 722141bfa6 Added a jobspec directive to specify envvars. Updated Docker driver to use them accordingly. 2015-10-02 17:46:55 -07:00
Alex Dadgar e60e6ff8f5 Add debug line when setting link speed to default 2015-10-02 17:32:17 -07:00
Alex Dadgar 4cc8588cb5 Merges 2015-10-02 17:32:11 -07:00
Alex Dadgar faf8002d2d go-fmt 2015-10-02 10:54:04 -07:00
Alex Dadgar 0c7f3e21d7 Merge pull request #199 from achanda/double_call 
Fix two issues where settings were not taking effect
 2015-10-02 10:52:41 -07:00
Abhishek Chanda 3751b46673 Make sure createHostConfig is called only once 
The second call discards any mods to port binding or
network type

Fixes #185 and #196
 2015-10-02 10:43:37 -07:00
Antoine POPINEAU f35ebe14ad Merge branch 'master' into dev/net-fingerprint 2015-10-02 09:38:35 +02:00
Antoine POPINEAU 36daef1625 Renamed all instances of Iface to NetworkInterface. 2015-10-02 09:29:18 +02:00
Alex Dadgar 191ea91cdd Merge pull request #184 from achanda/docker_net 
Enable setting networking mode for docker
 2015-10-01 19:32:48 -07:00
Abhishek Chanda 0cb649bd5f Enable setting networking mode for docker 
This patch enables setting networking mode for the docker
driver. This does not handle the `container` mode.
Closes #175
 2015-10-01 18:52:41 -07:00
Antoine POPINEAU 3a4b304a53 Refactored code. 2015-10-01 21:16:39 +02:00
Antoine POPINEAU 2e1488fb04 Comment some undocumented code. 2015-10-01 20:39:10 +02:00
Ryan Uber 836fd8a015 Merge pull request #153 from aldergren/master 
Prevent df on linux from adding line breaks to output
 2015-10-01 11:21:02 -07:00
Antoine POPINEAU b01912abdc Forgot some error checking. 2015-10-01 17:33:40 +02:00
Antoine POPINEAU 70d34da1bc Client config param added to specify net iface to use for fingerprinting. Added a Golang-native method for determining the interface IP address. 2015-10-01 17:31:47 +02:00
Orivej Desh 443d2deb32 Update obsolete comments 2015-09-29 20:33:11 +00:00
Orivej Desh d844997661 Fix "ifconfig not found" warning 2015-09-29 20:06:37 +00:00
Niklas Aldergren 9145d8ffb9 Prevent df on linux from adding line breaks to output 2015-09-29 13:23:41 +02:00
Orivej Desh 4ecf7a0f74 Improve link speed fingerprint error reporting 
With disconnected eth0 log output now looks like:

[WARN] fingerprint.network: Unable to parse IP in output of '/bin/ifconfig eth0'
[WARN] fingerprint.network: Unable to parse Speed in output of '/usr/sbin/ethtool eth0'

In a virtual machine:

[WARN] fingerprint.network: Unable to parse Speed in output of '/sbin/ethtool eth0'
[WARN] fingerprint.network: Unable to read link speed from /sys/class/net/eth0/speed
 2015-09-28 23:34:25 +00:00
Mitchell Hashimoto 690634d245 client: fix windows build errors 2015-09-27 23:53:25 -07:00
Armon Dadgar 6ff19853ef docker: check for missing unix socket 2015-09-27 17:49:58 -07:00
Alex Dadgar d314b514f8 Better Sprintf 2015-09-27 17:41:43 -07:00
Alex Dadgar f8e434aa1d Add environment package 2015-09-27 17:37:25 -07:00
Alex Dadgar 7ba9681a94 Fix args in test 2015-09-27 17:37:24 -07:00
Alex Dadgar a4a440da8e ParseAndReplace args at the executor level 2015-09-27 17:37:24 -07:00
Alex Dadgar 76dfa9bb0f Check err 2015-09-27 17:37:24 -07:00
Alex Dadgar b95d349a3e Set the allocdir correctly when using the linux executor 2015-09-27 17:37:24 -07:00
Alex Dadgar a45f9f2e71 Pull task environments into their own type to be reusable across drivers and executors 2015-09-27 17:37:24 -07:00
Alex Dadgar a96f6dfa6d Remove cruft from rebase 2015-09-27 17:37:24 -07:00
Alex Dadgar a7eebcbd5b merge fixes 2015-09-27 17:37:24 -07:00
Chris Bednarski 162247ffbe Correct implementation of config check for container and image cleanup; moved into Start and Open because handle does not have the config object 2015-09-27 13:59:38 -07:00
Chris Bednarski 9455370d8e Merge branch 'master' into f-docker-config-cleanup 2015-09-26 21:52:02 -07:00
Chris Bednarski 5678859752 Add configuration option for whether Nomad will clean up containers and images when they are no longer used 2015-09-26 18:53:15 -07:00
Chris Bednarski 79488baa4d Add all containers when searching for images that are still in used, since the container using the image may be stopped 2015-09-26 18:51:36 -07:00
Chris Bednarski af314b0f4d Log an info when an image can't be removed because it's still in use 2015-09-26 00:34:57 -07:00
Chris Bednarski 5632a33036 Use env.Get, remove sha256 checks, and change from regexp to strconv to test for int 2015-09-25 23:55:01 -07:00
Chris Bednarski 7021ac3ce8 Add docker 'command' to docs; cleanup some comments 2015-09-25 23:28:23 -07:00
Chris Bednarski 23407c54af Replace docker version and docker ps with native calls; no more exec! 2015-09-25 23:13:40 -07:00
Chris Bednarski 72cd22347f Docker kill now exits with an error, as expected 2015-09-25 22:43:19 -07:00
Chris Bednarski 0c31a83783 Go native for docker API calls 2015-09-25 20:01:03 -07:00
Chris Bednarski 198159c5be Put that back! 2015-09-25 18:38:41 -07:00
Chris Bednarski a8c8fc2f96 Merge branch 'master' into b-docker-fixes 
Conflicts:
	client/driver/docker_test.go
	client/driver/java_test.go
	client/driver/qemu_test.go
 2015-09-25 18:35:23 -07:00
Chris Bednarski efb7c409bb Improve handling of tagged docker images 
- Converted Start() to use docker native client throughout
- Better handling of latest vs. tagged images
    - We can now support multiple versions of an image
    - We will now not download a tag if we already have it
 2015-09-25 18:22:10 -07:00
Alex Dadgar 6ca7c6680d Fix client init logging and test 2015-09-25 18:12:11 -07:00
Alex Dadgar 70a1a37f84 Small test cleanup 2015-09-25 17:55:29 -07:00
Chris Bednarski d8e492d950 Added a failing test case for multiple docker tags 2015-09-25 17:38:07 -07:00
Chris Bednarski aa3019780d Added docker test to spin up 3 containers at the same time 2015-09-25 17:35:33 -07:00
Chris Bednarski 32d260281f Added guards to tests so we can more gracefully degrade the test suite when thing are not installed 2015-09-25 16:49:26 -07:00
Alex Dadgar 5a76693bc6 Fix comments 2015-09-25 16:49:19 -07:00
Alex Dadgar 3cf6d014ef Log the alloc and state directory on Client Init() 2015-09-25 16:49:19 -07:00
Alex Dadgar 998a817a29 Use host file/dir permissions when embedding 2015-09-25 16:49:19 -07:00
Alex Dadgar 3534839367 Destroy allocdirs in task runner tests 2015-09-25 16:49:19 -07:00
Alex Dadgar 5928b950c5 Skip non-regular files in embedding and comment on the chrootEnv map 2015-09-25 16:49:19 -07:00
Alex Dadgar 530e2d3c6d Change allocdir variable name for clarity 2015-09-25 16:49:19 -07:00
Alex Dadgar 8d7ededad7 Windows shouldn't do anything on unmount 2015-09-25 16:49:19 -07:00
Alex Dadgar eb2f01fd94 Update CPU limit to be an int 2015-09-25 16:49:18 -07:00
Alex Dadgar fc2e9754a3 Only unmount once 2015-09-25 16:49:18 -07:00
Alex Dadgar 0f855e2088 Fix cgroup isolation 2015-09-25 16:49:18 -07:00
Alex Dadgar e594e6081b Fix embed non-existent test 2015-09-25 16:49:18 -07:00
Chris Bednarski ac70832eb3 Added errNoResources to executor package; return this when Limit is called with nil argument 2015-09-25 16:49:18 -07:00
Chris Bednarski b9dd4c40c5 Added basic resources to exec/java tests so each test has cpu, ram, and network 2015-09-25 16:49:18 -07:00
Alex Dadgar 3cea4288b9 Merge qemu test 2015-09-25 16:49:14 -07:00
Alex Dadgar 6725cbb3f5 Mount shared alloc dir, modified API and tests 2015-09-25 16:46:41 -07:00
Alex Dadgar 81ade0f5d3 Linux Embed and AllocDir unit tests 2015-09-25 16:46:41 -07:00
Alex Dadgar 6879e59494 Fix function header 2015-09-25 16:46:41 -07:00
Alex Dadgar c9cffb93f6 Embed interface 2015-09-25 16:46:41 -07:00
Alex Dadgar 72d892a518 Client init rebase 2015-09-25 16:46:41 -07:00
Alex Dadgar 2e8395c458 Changed file names so it builds correctly 2015-09-25 16:46:41 -07:00
Alex Dadgar 9d3e3c0704 AllocDirBuilder that creates the alloc directory structure 2015-09-25 16:46:41 -07:00
Armon Dadgar 5bc26c3f9d Merge pull request #101 from hashicorp/f-qemu-network-update 
driver/qemu: Use Restricted Ports, additional port forwarding
 2015-09-25 16:36:46 -07:00
Clint Shryock f1ede2f555 Fix bad copy-paste from refactoring 2015-09-25 08:36:46 -05:00
Ryan Uber 6f15160f5c client: support modifying the servers list 2015-09-24 20:16:47 -07:00
Chris Bednarski fce6a5f7c7 Merge pull request #108 from hashicorp/b-netmap-crash 
Fix crash in task_runner tests
 2015-09-24 16:27:54 -07:00
Chris Bednarski a589c056ea All the TestTaskRunner* tests want this 2015-09-24 16:00:36 -07:00
Chris Bednarski e6ac055c6c Initialize reserved ports 2015-09-24 15:55:41 -07:00
Chris Bednarski d2aa4fd53c Merge pull request #104 from hashicorp/f-docker-static-ports 
Add static (reserved) ports for docker
 2015-09-24 15:44:41 -07:00
Chris Bednarski f032e311d2 Missed a set of parens 2015-09-24 15:40:32 -07:00
Chris Bednarski c9482e1188 Initialize reserved ports in task_runner 2015-09-24 15:37:43 -07:00
Chris Bednarski 40ce6ece41 Get only one version of a docker container 2015-09-24 14:07:28 -07:00
Clint Shryock 4e6c795352 re-order the resources check and don't bail if no ports requested 2015-09-24 14:32:54 -05:00
Chris Bednarski 3fd3c2a913 Add test case for PopulateEnvironment 2015-09-24 00:17:33 -07:00
Chris Bednarski f8e8084c71 Rename containerConfigForTask to createContainer to mirror the docker api call; other helper functions will follow this convention too 2015-09-24 00:00:46 -07:00
Chris Bednarski a071bb6d59 Rename containerConfig to hostConfig since there are two types of container configs 2015-09-23 23:57:04 -07:00
Chris Bednarski 9ff0e88d33 Added static / reserved port allocations to docker 2015-09-23 23:45:34 -07:00
Clint Shryock e67d892ea3 driver/qemu: Setup multiple port forwards 2015-09-23 23:15:28 -05:00
Chris Bednarski 1978be24d0 Format as string instead of casting to string 2015-09-23 19:29:53 -07:00
Chris Bednarski e413c6f9d2 Support mapped ports and labelled ports for docker 2015-09-23 18:19:01 -07:00
Chris Bednarski 1654d08fca Add docker ports; change to uppercase examples and tests 2015-09-23 18:01:08 -07:00
Chris Bednarski 8b2f8efe70 Merge branch 'master' into f-port-configs 2015-09-23 15:31:00 -07:00
Chris Bednarski b8fd345128 Docs and remove uppercase since it's checked at the config parse step 2015-09-23 15:28:58 -07:00
Armon Dadgar 3bdae950c5 Merge pull request #97 from hashicorp/f-qemu-driver-docs 
Basic Qemu docs
 2015-09-23 14:37:56 -07:00
Armon Dadgar 8aa2c292a0 Merge pull request #77 from hashicorp/f-network-fingerprint 
Network Fingerprinting
 2015-09-23 14:36:29 -07:00
Clint 15477ce433 Merge pull request #95 from hashicorp/f-java-driver-docs 
Add some basic docs for the Java Driver
 2015-09-23 15:59:33 -05:00
Clint Shryock e2c1bfe19c Basic Qemu docs 2015-09-23 14:44:49 -05:00
Chris Bednarski 73c4c1f210 Changed fmt for CPU to integer 2015-09-23 12:19:46 -07:00
Clint Shryock 0db8ce12d4 Update Java docs on how Java is detected 2015-09-23 14:16:43 -05:00
Clint Shryock 491f2d927f clean up some log formatting 2015-09-23 14:09:55 -05:00
Clint Shryock b93518b509 start the Qemu docs 2015-09-23 13:58:42 -05:00
Chris Bednarski e2c324d098 Merge branch 'master' into f-port-configs 2015-09-23 11:57:12 -07:00
Chris Bednarski f4ad6e51dc Replaced List with Map 2015-09-23 11:56:36 -07:00
Armon Dadgar b213462cb4 Change CPU from float64 to int 2015-09-23 11:14:32 -07:00
Clint Shryock 8c0542a9af Refactor network fingerprinting to be generic, use build flags 2015-09-23 10:12:56 -05:00
Clint Shryock 61d695b7d4 prefix warn/error messages 2015-09-23 10:03:35 -05:00
Clint Shryock e4a8565f00 fix casing for ERR logs 2015-09-23 10:00:06 -05:00
Chris Bednarski e26dbf4850 Fix some printf tokens 2015-09-22 23:43:05 -07:00
Chris Bednarski 9067b2594f Added environment variables to docker 2015-09-22 23:12:24 -07:00
Chris Bednarski 8bf45688b6 Check for nils before accessing data 2015-09-22 23:11:55 -07:00
Chris Bednarski 27376dc548 Added environment variables to Java 2015-09-22 22:36:10 -07:00
Chris Bednarski 3e90379f05 Pass dynamic ports to the exec driver via environment variables 2015-09-22 22:33:29 -07:00
Armon Dadgar fe72c8ea59 client: reset response struct to avoid decode errors 2015-09-22 22:10:28 -07:00
Armon Dadgar b7f370ba07 client: create dir, handle not exist more gracefully 2015-09-22 22:00:24 -07:00
Clint Shryock c84d51a9d1 don't re-convert mbits 2015-09-22 23:57:24 -05:00
Clint Shryock f876ed6673 refactoring, docs 2015-09-22 23:51:56 -05:00
Armon Dadgar ef81b29d7a client: fixing stats formating 2015-09-22 21:48:42 -07:00
Clint Shryock 088bc9e482 fix range error 2015-09-22 23:25:12 -05:00
Clint Shryock f60bbbfd15 populate node network resource 2015-09-22 23:22:23 -05:00
Clint Shryock e3c8ca8462 Update code for parsing IP address 2015-09-22 22:56:31 -05:00
Chris Bednarski 09249d877c Removed comment that's no longer relevant 2015-09-22 20:11:07 -07:00
Clint Shryock c615c560d2 Move ec2InstanceSpeed to a package variable, convert to string:int map 2015-09-22 22:04:20 -05:00
Alex Dadgar b8e7d28a06 Update package import name 2015-09-22 18:48:42 -07:00
Alex Dadgar dd2f16123f Fix qemu check and add debug lines when a driver is disabled 2015-09-22 17:36:44 -07:00
Alex Dadgar bbb3853dd8 Update Qemu Fingerprint check 2015-09-22 17:12:52 -07:00
Alex Dadgar e095664c49 Guard tests 2015-09-22 17:10:03 -07:00
Alex Dadgar 3875e8a59a Update exec_linux to assume it is running as root 2015-09-22 16:44:11 -07:00
Alex Dadgar dd0a76a9d7 Disable exec/java/qemu when non-root on non-windows OSes 2015-09-22 16:32:05 -07:00
Clint Shryock 5c749e1ffb Consolidate the AWS fingerprinters 2015-09-22 16:56:04 -05:00
Clint Shryock 2bbf080666 Refactor some AWS things, removing fingerprint/network.go 
- move isAWS to env, use in other places
- fingerprint/network.go is no empty; removed
 2015-09-22 16:31:57 -05:00
Clint Shryock b44c911996 change the naming 2015-09-22 16:01:29 -05:00
Chris Bednarski 168c959497 Added named ports 2015-09-22 13:59:16 -07:00
Clint Shryock a9cf7381ca Refactor the Network Fingerprinters 2015-09-22 15:59:00 -05:00
Clint Shryock f048326300 Rework client/fingerprint/fingerprint.go to use a slice and enforce ordering 2015-09-22 14:23:48 -05:00
Armon Dadgar ed2fc3a69f client: persist the node ID 2015-09-22 10:31:47 -07:00
Clint Shryock d11bd582f7 Basic network fingerprinting for Unix type, AWS systems 2015-09-22 11:44:53 -05:00
Alex Dadgar 1b46cbff83 Fixed function comment 2015-09-21 09:08:57 -07:00
Alex Dadgar 8d24e5e532 Tests skip based on cgroups, not root 2015-09-21 09:08:57 -07:00
Alex Dadgar e694688527 Update UniversalExecutor to have string identifiers 2015-09-21 09:08:57 -07:00
Alex Dadgar 0e3f21b34f Linux executor with cgroup isolation support 2015-09-21 09:08:57 -07:00
Clint Shryock 61948802a8 executor/linux: Add UUID for the name, return error on failure to apply 2015-09-21 09:08:57 -07:00
Clint Shryock 067e65bf12 exec/linux: move Limit to after Start 2015-09-21 09:08:57 -07:00
Clint Shryock 0c526ddb8e spike on adding cgroups 2015-09-21 09:08:57 -07:00
Armon Dadgar 807edc194d Merge branch 'f-docs' 2015-09-20 19:25:25 -07:00
Armon Dadgar 67fc3c9b87 client: optimize initial heartbeat time 2015-09-20 17:02:12 -07:00
Chris Bednarski d4948e2935 Windows does not support exec.Cmd.SysProcAttr.Credential call; this causes cross-compiltion to fail so it's behind a build flag now 2015-09-18 18:59:42 -07:00
Chris Bednarski c99e368461 Support running Linux exec as a non-root user 2015-09-15 20:17:23 -07:00
Chris Bednarski 57e4f7c44f Merge branch 'master' into f-executor 2015-09-15 19:03:52 -07:00
Chris Bednarski e5cb8852af Overhaul the executor code so we use build flags instead of dynamic loading 2015-09-15 18:54:55 -07:00
Chris Bednarski d1c4abef33 Changed Default executor to use a factory 2015-09-15 16:40:40 -07:00
Chris Bednarski 4cefbc3c22 Rename exec package to executor so it works better with goimports; fixes for Java driver 2015-09-15 14:03:03 -07:00
Chris Bednarski 6a7b82f7ba Update Java driver to use Executor 2015-09-15 13:45:48 -07:00
Chris Bednarski 9db88f6013 Updated exec driver to use nomad/client/exec 2015-09-15 13:11:56 -07:00
Chris Bednarski 06073b171d Use Linux on Linux 2015-09-14 19:38:21 -07:00
Chris Bednarski 6eb3d86528 Run as nobody if no user is specified 2015-09-14 19:04:29 -07:00
Chris Bednarski e4d3aad44e Send signal 0 to check whether the process is alive 2015-09-14 16:16:56 -07:00
Chris Bednarski 0425ae941c Implemented Linux driver with RunAs functionality -- added errors to RunAs and Limit 2015-09-14 15:57:21 -07:00
Armon Dadgar 2ff133c0e6 nomad: rename region1 to global. Fixes #41 2015-09-13 18:18:40 -07:00
Armon Dadgar d1f829e502 client: Use Alloc.TaskResouces to override Task.Resources 2015-09-13 17:35:56 -07:00
Ryan Uber 3d31230ac1 client: always create alloc dir if it is non-empty 2015-09-13 12:14:12 -07:00
Ryan Uber 58216245ac client: skip init in dev mode 2015-09-12 13:16:27 -07:00
Ryan Uber 823276993c client: init the alloc dir 2015-09-12 11:47:44 -07:00
Ryan Uber 1ff724ab25 client: alloc dirs tolerate missing directories 2015-09-11 20:32:55 -07:00
Chris Bednarski 91194d9337 Update the interface, add UniversalExecutor, add stub for LinuxExecutor 2015-09-11 17:12:48 -07:00
Chris Bednarski f511eeac0e Added stub / spec for executor interface 2015-09-11 12:35:03 -07:00
Chris Bednarski da93d4a30f Change error to err to be consistent with other usage 2015-09-11 10:26:33 -07:00
Chris Bednarski 4f83bc7090 Fix typos 2015-09-10 11:35:03 -07:00
Chris Bednarski 2ccad3d6e8 Change panic to a more helpful message 2015-09-09 19:04:06 -07:00
Chris Bednarski 237bf5f68e Merge branch 'f-driver-config' into f-docker-resource-constraints 2015-09-09 18:47:35 -07:00
Chris Bednarski 2a40673a3a Relax Qemu version regexp to accomodate build identifier in the version command 2015-09-09 18:39:40 -07:00
Chris Bednarski cc4d361ee9 Updated Qemu and tests to use DriverContext 2015-09-09 18:38:52 -07:00
Chris Bednarski 0281debee4 Merge branch 'master' into f-driver-config 
Conflicts:
	client/driver/driver.go
 2015-09-09 18:19:19 -07:00
Chris Bednarski a695e311dc Replace logging and config with DriverContext, which allows us to expand the dependency injection without changing the interface 2015-09-09 18:06:23 -07:00
Chris Bednarski 1ff755f31a Merge pull request #24 from hashicorp/qemu-driver 
Basic Qemu driver implementation
 2015-09-09 17:43:13 -07:00
Chris Bednarski 9965d28a22 Add logger to the dockerHandle created by Open() 2015-09-09 13:43:28 -07:00
Chris Bednarski a731a84136 Cleanup based on PR feedback 
- Add validation that Resources.MemoryMB and Resources.CPU have non-zero values
- Change log calls to use logger whenever possible
- Change log format to add colon after driver.docker
 2015-09-09 13:35:10 -07:00
Clint Shryock 037ae6d6eb driver/qemu+java: Code cleanups 2015-09-09 14:38:42 -05:00
Clint Shryock bd0f92f977 driver/qemu: Actually use the requred Memory from the Task 2015-09-09 14:30:35 -05:00
Clint Shryock 2611523b3b driver/qemu: Require a memory resource on the task 2015-09-09 14:28:16 -05:00
Chris Bednarski ac277c5765 Change SIGINT to SIGTERM 2015-09-09 01:52:58 -07:00
Chris Bednarski b70b852662 Added preliminary resource constraints for docker; CPU and Memory 2015-09-09 01:08:31 -07:00
Clint Shryock 98c53162d2 driver/qemu: add to builtin drivers 2015-09-08 16:25:36 -05:00
Clint Shryock d2822a435d driver/qemu: Basic lifecycle management of Qemu images 2015-09-08 16:08:49 -05:00
Chris Bednarski 4eb8fc5188 Added config to drivers; needed for docker driver to get the socket endpoint 2015-09-08 12:43:02 -07:00
Clint Shryock c1f982a1d0 Merge branch 'master' into qemu-driver 
* master: (66 commits)
  Use a single implementation of GenerateUUID
  scheduler: use update strategy for rolling updates
  nomad: adding helper structs
  scheduler: Adding CreateEval to Planner
  nomad: worker supports create eval
  nomad: guard eval creation based on parent eval
  nomad: adding Eval.Create endpoint
  nomad: support time wait for evaluations
  nomad: adding fields to cause an evaluation to 'wait'
  scheduler: support in-place allocation updates
  scheduler: util method to diff task groups
  nomad: helper struct func
  nomad: PopUpdate should minimize plan size
  nomad: adding helper to manipulate plans
  scheduler: share context and stack
  scheduler: allow updating the base nodes
  scheduler: allow StaticIterator to update base set
  scheduler: pull node shuffle into util
  nomad: adding node GC
  nomad: using Raft StartAsLeader to make tests faster
  ...
 2015-09-08 09:52:40 -05:00
Chris Bednarski bd06e4bad2 Merge pull request #19 from hashicorp/test-resources 
Added example python and java apps for testing
 2015-09-07 17:03:09 -07:00
Armon Dadgar ea0795995d Use a single implementation of GenerateUUID 2015-09-07 15:23:03 -07:00
Armon Dadgar 7d69aa78c1 nomad: using Raft StartAsLeader to make tests faster 2015-09-07 10:46:41 -07:00
Armon Dadgar 8bedd3769c nomad: unifying the state store API 2015-09-06 20:56:38 -07:00
Armon Dadgar 1a5579384a nomad: cleanup API descrepencies 2015-09-06 20:47:42 -07:00
Armon Dadgar 0702f17989 Rename client endpoint to node endpoint 2015-09-06 20:31:32 -07:00
Armon Dadgar c7773feced nomad: improve error messages at start for dev mode 2015-09-06 20:18:47 -07:00
Armon Dadgar f1da1226c1 driver/java: Add to builtin list 2015-09-04 13:09:11 -07:00
Clint Shryock 60ad3b7517 Merge remote-tracking branch 'upstream/master' into qemu-driver 
* upstream/master:
  Removed debug logging
  Added tests for docker, improved logging, and fixed a few bugs
  Added initial implementation for the docker driver; tests pending
 2015-09-04 14:50:47 -05:00
Chris Bednarski aa3937f248 Added example python and java apps for testing 
- Java example from @catsby
 2015-09-04 11:13:24 -07:00
Chris Bednarski 813f905a1c Removed debug logging 2015-09-03 21:08:14 -07:00
Chris Bednarski c483036e16 Added tests for docker, improved logging, and fixed a few bugs 2015-09-03 21:00:16 -07:00
Clint Shryock 9cb1dc95a0 driver: Add README.md for docs 2015-09-03 17:01:43 -05:00
Chris Bednarski b9ceb69683 Added initial implementation for the docker driver; tests pending 2015-09-03 12:35:22 -07:00
Clint 2000d3657f Merge pull request #15 from hashicorp/b-java-driver 
Rough spike at adding Java driver
 2015-09-03 14:31:49 -05:00
Clint Shryock 8c6128503f driver/java: use filepath.Join instead of simple + 2015-09-03 10:25:09 -05:00
Clint Shryock 256d5c1286 driver/java: Don't error if Java isn't found; log if error parsing 2015-09-03 10:02:48 -05:00
Clint Shryock 1dd0b63cb4 clean up orphaned java processes 2015-09-03 09:49:39 -05:00
Clint Shryock c774c67b6b driver/java: update Fingerprint for Java driver 2015-09-02 11:41:25 -05:00
Clint Shryock 9d5c04ddbb Spike at a Java driver, runs Java jars 2015-09-01 17:07:41 -05:00
Clint Shryock 9e79a15789 move test logger to shared file 2015-09-01 16:56:42 -05:00
Chris Bednarski 7070b80167 Added test for config.Read and config.ReadDefault 2015-08-31 20:11:01 -07:00
Chris Bednarski 2e5a080354 Delete unused NewConfig function 2015-08-31 20:01:04 -07:00
Chris Bednarski d9d78d6b2c Add working test for consul fingerprinter 2015-08-31 19:56:38 -07:00
Chris Bednarski ab0deab3e5 Made consul fingerprinter configurable 2015-08-31 19:56:25 -07:00
Chris Bednarski ecfeba3ce6 Added ReadDefault for supreme laziness 2015-08-31 19:54:49 -07:00
Chris Bednarski 1936932406 Added Config.Options so we can support arbitrary key-value configuration 2015-08-31 19:48:59 -07:00
Chris Bednarski ff5266ad8f Add consul data to node.Links 2015-08-31 16:04:48 -07:00
Chris Bednarski a033bedbaf Added basic consul functionality; currently hard-coded to a test instance 2015-08-31 16:04:48 -07:00
Clint 05682391b9 Merge pull request #14 from hashicorp/aws-env 
Spike on AWS ENV fingerprint
 2015-08-31 15:10:33 -05:00
Clint Shryock 1e652cc4e9 fingerprint/env_aws: Rename ec2-instance link to aws.ec2, drop ami link 2015-08-31 15:02:31 -05:00
Clint Shryock 416e7de06d fingerprint/aws: Add 2 basic links, rename 'env' to 'platform' 2015-08-31 14:18:40 -05:00
Armon Dadgar 0527529f88 client: allocate alloc dir and cleanup 2015-08-30 17:35:58 -07:00
Armon Dadgar ab069de9b9 client: more stats 2015-08-30 17:24:12 -07:00
Armon Dadgar 5549a82f9f client: testing state restore 2015-08-30 17:19:20 -07:00
Armon Dadgar 43471d023e client: test alloc runner 2015-08-30 17:10:17 -07:00
Armon Dadgar 7b3adc6cc4 client: refactor AllocRunner for testing 2015-08-30 16:35:04 -07:00
Armon Dadgar 8c25d1fca1 client: testing task runner 2015-08-29 20:07:24 -07:00
Armon Dadgar db33f76a61 client: remove TaskRunner dependence on AllocRunner 2015-08-29 19:42:35 -07:00
Armon Dadgar a2d8c6c3fe client: working on state restore 2015-08-29 19:14:47 -07:00
Armon Dadgar b6332b435d client: first pass at save/restore of state 2015-08-29 18:16:49 -07:00
Armon Dadgar 50c677a9bb client: adding state save helpers 2015-08-29 18:03:00 -07:00
Armon Dadgar c99fe72f54 driver/exec: basic implementation 2015-08-29 16:20:07 -07:00
Armon Dadgar 83dd313b5c client: working on runners 2015-08-29 15:46:10 -07:00
Armon Dadgar e57ee3fbfe client: test watching allocations 2015-08-29 14:33:30 -07:00
Armon Dadgar 4d6238ebb8 client: test updating alloc status 2015-08-29 14:22:24 -07:00
Armon Dadgar 2e4809d2fc client: test heartbeats 2015-08-29 14:15:34 -07:00
Clint Shryock ae1142ad33 cleanups 2015-08-28 13:09:21 -05:00
Clint Shryock dc217da84a spike on AWS ENV fingerprint 2015-08-28 11:33:41 -05:00
Chris Bednarski 96cb220ff4 Update references to "os" to use "kernel.name" 
This brings test code and mocks up to date with the fingerprinter. This was a slightly larger change than I anticipated, but I think it's good for two reasons:

1. More semanitcally correct. `os.name` is something like "Windows 10 Pro" or "Ubuntu", while `kernel.name` is "windows" or "linux". `os.version` and `kernel.version` match these semantics.
2. `kernel.name` is much easier to grep for than `os`, which is helpful because oracle can't help us with strings.
 2015-08-28 01:30:47 -07:00
Chris Bednarski 9b8fc07bbe Drop os fingerprint since this is redundant with host 2015-08-27 17:37:56 -07:00
Chris Bednarski 94235e3448 Change 'os' to 'kernel.name'; add 'kernel.version' for *nix* 2015-08-27 17:37:37 -07:00
Chris Bednarski ced2f647c6 Add guard to storage fingerprint so CWD will be used if AllocDir is not specified -- fixes windows tests 2015-08-27 16:03:09 -07:00
Chris Bednarski d0a091c952 Add the storage fingerprinter to the list of built-ins 2015-08-27 14:30:41 -07:00
Chris Bednarski 0437a02fd7 Change column constraint so this is compatible with darwin and linux 2015-08-27 14:30:20 -07:00
Clint c6cdf014fd Merge pull request #7 from hashicorp/memory-fingerprint 
Add a memory fingerprint
 2015-08-27 16:11:44 -05:00
Clint Shryock b223278b6f convert to MB for MemoryMB, and update test 2015-08-27 16:01:47 -05:00
Chris Bednarski 6123ba8b83 Get average frequency of all CPUs so we can do average frequency * cores for total compute 2015-08-27 13:35:54 -07:00
Clint Shryock 759d3131f2 Merge remote-tracking branch 'upstream/master' into memory-fingerprint 
* upstream/master:
  Prevent a panic if the df tool output is not in the expected format
  Change 'available' to 'free' because it's shorter; also initialize node.Resources.DiskMB as total disk instead of available disk
  Changed logs to errors; added data to node.Resources.DiskMB
  populate CPU in Node Resources
  Fix arity in log call for TCP resolution failure
  Add storage fingerprinter
 2015-08-27 15:30:00 -05:00
Clint e3348ad809 Merge pull request #6 from hashicorp/cpu-resources 
populate CPU in Node Resources
 2015-08-27 15:26:00 -05:00
Clint Shryock 83a792593e Add a memory fingerprint, just grabs memory 2015-08-27 15:19:19 -05:00
Chris Bednarski 78767f0ff4 Merge pull request #4 from hashicorp/f-storage-fingerprint 
Add storage fingerprinter
 2015-08-27 12:43:18 -07:00
Chris Bednarski 8e544e0969 Prevent a panic if the df tool output is not in the expected format 2015-08-27 12:42:58 -07:00
Chris Bednarski 66b518cd6c Change 'available' to 'free' because it's shorter; also initialize node.Resources.DiskMB as total disk instead of available disk 2015-08-27 12:37:05 -07:00
Chris Bednarski d17c6128c8 Changed logs to errors; added data to node.Resources.DiskMB 2015-08-27 12:23:17 -07:00
Clint Shryock 58be8ba139 populate CPU in Node Resources 2015-08-27 14:15:56 -05:00
Clint Shryock cacb8074d3 Add cpu.frequency, cpu.totalcompute 2015-08-27 09:19:53 -05:00
Chris Bednarski cd5cd7efc2 Fix arity in log call for TCP resolution failure 2015-08-26 17:17:08 -07:00
Chris Bednarski edea3700a4 Add storage fingerprinter 
Breaking change: Added ID() to Fingerprint interface. This allows us to assign
each fingerprint implementation a unique ID, identify which fingerprinters have
been run, and also self-identify in log messages.

- Added storage fingerprinter for Windows and *nix
- Added storage tests under storage_test.go
- Added test helper functions under fingerprint/fingerprint_test.go
- Added ID() to existing finterprinters and drivers
- Added Fingerprint.ID() to log messages via log.SetPrefix()
 2015-08-26 17:17:08 -07:00
Clint Shryock 9a92383b69 refactor into multiple fingerprints 2015-08-26 16:29:18 -05:00
Clint Shryock 8ab31da8b0 update fingerprint with more cpu, some host info 2015-08-26 15:20:07 -05:00
Clint Shryock d2318f71f1 Fingerprint CPU model name, count of cores 2015-08-26 11:32:30 -05:00
Chris Bednarski 0b255c6af6 Pass config to fingerprints; update tests 2015-08-25 16:21:29 -07:00
Chris Bednarski 98331bbf87 We need a new package for this or we get a circular import between client and client/fingerprint 2015-08-25 16:13:33 -07:00
Armon Dadgar 265da591ae client: faster retries in dev mode 2015-08-23 17:40:14 -07:00
Armon Dadgar 07b832c588 agent: use the client 2015-08-23 16:53:15 -07:00
Armon Dadgar 8e32075210 client: making progress 2015-08-23 16:49:48 -07:00
Armon Dadgar 3663c44dee client: standardize naming 2015-08-23 15:36:06 -07:00
Armon Dadgar d7c785aad4 client: rename Alloc/TaskContext to Runner 2015-08-23 15:32:46 -07:00
Armon Dadgar 7cd52e10bc client: alloc/task context 2015-08-23 15:30:16 -07:00
Armon Dadgar c57821877b client: working on alloc context 2015-08-23 15:15:48 -07:00
Armon Dadgar 0f3ef2a48a client: working on alloc handling 2015-08-23 15:06:47 -07:00
Armon Dadgar c71c9bec1a client: working with alloc diffs 2015-08-23 14:54:52 -07:00
Armon Dadgar 1dfa7296c1 client: alloc diffing 2015-08-23 14:47:51 -07:00
Armon Dadgar 811d6d85e1 client: state stubs 2015-08-23 14:12:26 -07:00
Armon Dadgar 5705dc6398 client: watch for allocations 2015-08-22 19:31:22 -07:00
Armon Dadgar 6c8791ccf8 nomad: client does periodic heartbeat 2015-08-22 18:16:05 -07:00
Armon Dadgar 2b2e4c2256 client: register on start 2015-08-20 17:49:04 -07:00
Armon Dadgar c5553017be client: scan for drivers 2015-08-20 16:53:43 -07:00
Armon Dadgar a561fbb9a9 driver: skeleton package 2015-08-20 16:50:28 -07:00
Armon Dadgar d44bf839ef client: attempt fingerprint of host 2015-08-20 16:41:29 -07:00
Armon Dadgar 88438619c8 fingerprint: adding arch fingerprint 2015-08-20 16:29:19 -07:00
Armon Dadgar f20b361d89 fingerprint: adding simple package 2015-08-20 16:26:32 -07:00
Armon Dadgar a6c4d2a63f client: testing RPC 2015-08-20 16:13:05 -07:00
Armon Dadgar 0f2fe6a425 client: testing RPC 2015-08-20 16:12:28 -07:00
Armon Dadgar 7c3e987617 client: skeleton package 2015-08-20 16:07:26 -07:00
Armon Dadgar 0383bf3c14 client: adding skeleton 2015-08-20 15:25:09 -07:00