luxolus/open-nomad
2
0
Fork 0
Code Issues 1 Pull requests Packages Releases Wiki Activity
23446 commits 1 branch 0 tags 311 MiB
Commit graph

164 commits

Author SHA1 Message Date
Alex Dadgar d760e68774 darwin test fixes 2017-07-22 19:48:47 -07:00
Michael Schurter 420be86e39 Test AllocDir.Copy 2017-07-17 15:46:54 -07:00
Michael Schurter 51515cbe0c Ensure allocDir is never nil and persisted safely 
Fixes #2834
 2017-07-17 15:46:54 -07:00
Mark Mickan c196d320f8 Add tests for migrating symlinks in alloc and local directories 2017-06-04 15:56:22 +09:30
Mark Mickan 236f24c9a4 Include symlinks in snapshots when migrating disks 
Fixes #2685
 2017-06-04 00:36:18 +09:30
Michael Schurter e204a287ed Refactor Consul Syncer into new ServiceClient 
Fixes #2478 #2474 #1995 #2294

The new client only handles agent and task service advertisement. Server
discovery is mostly unchanged.

The Nomad client agent now handles all Consul operations instead of the
executor handling task related operations. When upgrading from an
earlier version of Nomad existing executors will be told to deregister
from Consul so that the Nomad agent can re-register the task's services
and checks.

Drivers - other than qemu - now support an Exec method for executing
abritrary commands in a task's environment. This is used to implement
script checks.

Interfaces are used extensively to avoid interacting with Consul in
tests that don't assert any Consul related behavior.
 2017-04-19 12:42:47 -07:00
Michael Schurter 4b18f916d9 Set ownership on directories in chroot 
Also support getOwner on all Unixes as they all have `Stat_t.{U,G}id`
 2017-04-17 12:41:33 -07:00
Michael Schurter 83fb374d0b Don't disable hardlinking! 2017-04-17 11:03:15 -07:00
Michael Schurter 547d5a81b4 Chown files when copying into chroot 
Fixes #2552

Not needed when hardlinking. Only adds Linux support but other OS's may
be easy.
 2017-04-17 11:03:15 -07:00
Alex Dadgar f2ba6db607 Add sticky bit to temp directory 
Fixes an issue where the Ruby runtime expects the sticky bit to be set
on the temp directory. The sticky bit is commonly set on the temp
directory since it is usually shared by many users. This change brings
ours in line with that assumption.
 2017-04-04 10:48:29 -07:00
Alex Dadgar 3f68aae7ab TODO 2017-03-11 13:56:19 -08:00
James Nugent b3d1ef0af6 client/allocdir: Add missing functions on Solaris 
This commit adds Solaris versions of the following functions:

- `linkDir`
- `unlinkDir`
- `createSecretDir`
- `removeSecretDir`

I believe this requires Go 1.8 in order to compile, as the unlink
syscall was previously missing.
 2017-03-09 13:49:14 -05:00
Alex Dadgar 36f3788cb7 Fix lint errors 2017-03-02 21:03:05 -08:00
Alex Dadgar 4cba170e4b Mkdir -> MkdirAll to avoid error when folder already exists 2017-03-02 19:35:31 -08:00
Michael Schurter d12949fbbd Cleanup comments/names 2017-03-02 15:44:52 -08:00
Michael Schurter e03f64ea6a Safely ensure {dev,proc,alloc} are mounted 
If they're unmounted by a reboot they'll be properly remounted.
 2017-03-02 13:21:34 -08:00
Michael Schurter d5b7c0c302 unlinkDir should not error if already unlinked 2017-03-02 13:20:47 -08:00
Michael Schurter a9f9c485b4 Fix typos 2017-03-02 13:20:05 -08:00
Michael Schurter 1cfe9f88b8 Make sure unmounting the secrets dir is idemptoent 2017-03-01 17:19:39 -08:00
Michael Schurter e5a29b82db Fix allocdir Move test and make code more defensive 
A change in the behavior of `os.Rename` in Go 1.8 brought to light a
difference in the logic between `{Alloc,Task}Runner` and this test:

AllocRunner builds the alloc dir, moves dirs if necessary, and then lets
TaskRunner call TaskDir.Build().

This test called `TaskDir.Build` *before* `AllocDir.Move`, so in Go 1.8
it failed to `os.Rename over` the empty {data,local} dirs.

I updated the test to behave like the real code, but I defensively added
`os.Remove` calls as a subtle change in call order shouldn't break this
code. `os.Remove` won't remove a non-empty directory, so it's still
safe.
 2017-02-21 17:22:10 -08:00
Michael Schurter d8b4372904 Don't link shared alloc dir into task dir for raw_exec 
Fixes running raw_exec tasks when nomad isn't root.
 2017-01-18 11:28:34 -08:00
Michael Schurter c90cd0d874 Stop trying to use mount for image based drivers 
Fixes #2178 and allows using Docker and other image based drivers even
when nomad is run as a non-root user.

`client/allocdir` tests can be run as a non-root user to ensure this
behavior and tests that rely on root or non-root users properly detect
their effective user and skip instead of fail.
 2017-01-13 13:04:12 -08:00
Michael Schurter 86fcf96f72 Put a logger in AllocDir/TaskDir 2017-01-05 16:31:56 -08:00
Michael Schurter f43d3f074a Add comments to TaskDir 2017-01-05 16:31:55 -08:00
Michael Schurter 3ea09ba16a Move chroot building into TaskRunner 
* Refactor AllocDir to have a TaskDir struct per task.
* Drivers expose filesystem isolation preference
* Fix lxc mounting of `secrets/`
 2017-01-05 16:31:49 -08:00
Alex Dadgar 072ff1c3ee ensure file doesn't escape 2016-12-18 15:48:30 -08:00
Diptanu Choudhury d9f8e3a75a Fixed comments 2016-11-08 12:55:15 -08:00
Diptanu Choudhury 2132fbb68a Fixed permission issues on client 2016-11-08 10:57:29 -08:00
Alex Dadgar 5559300372 Change folder permission event when not root 2016-10-28 16:52:38 -07:00
Diptanu Choudhury 1098dc4aa3 Fixed alloc dir move tests 2016-10-26 15:17:57 -07:00
Alex Dadgar 4ae735c8ba Disallow fs to read secret directory 2016-10-24 11:14:05 -07:00
Michael Schurter 285e80ac0f Remove disk usage enforcement 
Many thanks to @iverberk for the original PR (#1609), but we ended up
not wanting to ship this implementation with 0.5.

We'll come back to it after 0.5 and hopefully find a way to leverage
filesystem accounting and quotas, so we can skip the expensive polling.
 2016-10-21 13:55:51 -07:00
Ben Barnard 83f647ed84 Replace "the the" with "the" in documentation and comments 2016-10-11 15:31:40 -04:00
Alex Dadgar d2837dec44 Do not allow path to escape the alloc dir for the FS commands 2016-10-03 14:58:44 -07:00
Diptanu Choudhury d50c395421 Getting snapshot of allocation from remote node (#1741) 
* Added the alloc dir move

* Moving allocdirs when starting allocations

* Added the migrate flag to ephemeral disk

* Stopping migration if the allocation doesn't need migration any more

* Added the GetAllocDir method

* refactored code

* Added a test for alloc runner

* Incorporated review comments
 2016-10-03 09:59:57 -07:00
Diptanu Choudhury 2b1d214b0d Avoiding copying files if they are already present in chrootw (#1753) 2016-09-27 11:43:27 -07:00
Diptanu Choudhury 12c7873db2 Closing files when files are removed 2016-09-23 22:17:53 -07:00
Diptanu Choudhury 589356fd55 Adding a snapshot endpoint on the client (#1730) 2016-09-21 21:28:12 -07:00
Alex Dadgar 6702a29071 Vault token threaded 2016-09-14 13:30:01 -07:00
Alex Dadgar eecef73302 syscall error 2016-09-02 15:00:46 -07:00
Alex Dadgar eef786dd9d Secret dir materialized in alloc/task directory 2016-09-02 12:44:05 -07:00
Alex Dadgar 2c8dd8bbd3 Revert "Introduce a Secret/ directory" 2016-09-01 17:23:15 -07:00
Alex Dadgar 9fa23e3536 Symlink on windows 2016-08-31 21:41:44 -07:00
Alex Dadgar 5d3b47e648 Address comments and reserve 2016-08-31 18:11:02 -07:00
Alex Dadgar 0626eb9619 environment variables 2016-08-31 13:56:11 -07:00
Alex Dadgar d59e14eed4 Interface + tests 2016-08-30 21:40:32 -07:00
Alex Dadgar 14b7126511 Secret dir, hello world 2016-08-29 15:41:52 -07:00
Alex Dadgar aaca0bdaf4 Make maxSize exported so that it is serialized 2016-08-28 17:48:35 -07:00
Ivo Verberk 57012e8d8c Monitor the complete alloc directory, not just the shared part. 2016-08-25 20:48:19 +02:00
Ivo Verberk 2a17895a83 Disk resource monitoring and enforcement 2016-08-18 07:59:03 +02:00
Alex Dadgar 8323b6a0b5 only use polling 2016-08-11 18:59:48 -07:00
Alex Dadgar 3ea95bb91c initial log api impl 2016-07-25 11:16:01 -07:00
Diptanu Choudhury 22af229cef Merge pull request #1321 from mwieczorek/f-windows-binds 
Volume binds for windows containers
 2016-07-18 10:20:44 -06:00
Alex Dadgar c8e7b909c7 Merge pull request #1404 from hashicorp/f-streaming 
Implement a streaming API and tail in the fs command
 2016-07-12 17:23:04 -06:00
Alex Dadgar 661d100f2f address comments 2016-07-12 17:01:33 -06:00
Sean Chittenden d309649ada
Darwin currently has allocdir support. 
Pointed out by: @dadgar
 2016-07-11 12:19:17 -07:00
Alex Dadgar e9ffadfdc6 initial comments 2016-07-11 10:58:18 -06:00
Sean Chittenden 2983bd6fce
Fix test for non-Linux platforms. 
The following tests now check a whitelist for whether or not their
driver is present or not, or if the OS is supported or not.

* `TestAllocDir_MountSharedAlloc`
* `TestClient_Drivers_InWhitelist` (`exec` driver)
* `TestClient_Drivers` (`exec` driver)
* `TestJavaDriver_Fingerprint` (`java` driver)
 2016-07-10 15:19:49 -07:00
Alex Dadgar 51ae7ace25 initial tail impl 2016-07-10 13:57:04 -04:00
Michal Wieczorek 67a04bb1cc Volume binds for windows containers 2016-06-20 21:46:33 +02:00
Sean Chittenden dc28ab0cb5
Speling police 2016-05-15 09:41:34 -07:00
Sean Chittenden 514f22c4a8 Merge pull request #1160 from hashicorp/f-freebsd 
Stubbed out raw_exec support for *NIX platforms

OOB LGTM: @diptanu
 2016-05-11 12:54:27 -07:00
Diptanu Choudhury 2f15842f3e Merge pull request #1164 from hashicorp/fix-unmount 
Removing directories only if we could successfully unmount them
 2016-05-11 12:51:34 -07:00
Diptanu Choudhury 5b12aebd31 Removing directories only if we could successfully unmount them 2016-05-10 14:57:16 -07:00
Sean Chittenden f2e01f0eab
Stub out FreeBSD support for Nomad 
Compiles, but is not functional (yet).
 2016-05-09 11:56:35 -07:00
Sean Chittenden 09f7d5e595 Prefer golang.org/x/sys/unix where appropriate 
Favor the `unix` package on *NIX platforms vs the now frozen `syscall` package.
 2016-05-07 11:01:45 -07:00
Sean Chittenden 1314227863 Explicitly enumaret the build targets in _unix 
`!windows` was being used as the synonym for `darwin dragonfly freebsd linux netbsd openbsd solaris`.  While I don't imagine `android` will be a prime target for Nomad in the near term, favor explicit build targets.

List of build targets generated by Go 1.7's dist command: `go tool dist list | sort | cut -d '/' -f 1 | sort | uniq`
 2016-05-07 10:42:01 -07:00
Sean Chittenden cfd76aaf15 Rename from posix to unix to parallel x/sys/unix 
Use the `_unix` file suffix to denote *NIX-like semantics in order to parallel the designation given by the package `golang.org/x/sys/unix`.
 2016-05-07 10:33:43 -07:00
Diptanu Choudhury 9288ac5117 Seeking to offset while doing readAt with non-zero offset 2016-04-04 13:05:02 -07:00
Alex Dadgar 2de8ab6b1b Alloc dir uses MkdirAll 2016-03-28 14:33:53 -07:00
Alex Dadgar 565bb655d6 Drop the permissions of sub directories in the alloc dir 2016-02-25 11:49:43 -08:00
Diptanu Choudhury 03c32d9bad Added a test for AllocDir.LogDir 2016-02-25 09:08:51 -08:00
Diptanu Choudhury 47da0b02b3 Writing logs to alloc dir 2016-02-24 20:06:43 -08:00
Alex Dadgar 75473e88d2 Unmount special directories when task finishes 2016-02-08 18:51:11 -08:00
Diptanu Choudhury e89bcf52b8 Moving code to mount and unmount chroot into allocdir 2016-02-08 14:11:53 -08:00
Alex Dadgar 66f59e2dc0 Add tmp file to each task directory 2016-02-04 15:35:04 -08:00
Alex Dadgar b45b101240 Serialize the list of mounted entries in the alloc dir 2016-02-04 14:30:32 -08:00
Ranjib Dey 4527257647 allow group and others to have executable permissions 2016-01-31 10:54:32 -08:00
Diptanu Choudhury 980bc19d10 Added more information about files 2016-01-27 14:20:10 -08:00
Alex Dadgar 1ceb6f012a Fix a bunch of tests 
Up timeouts

trusty travis beta

Increase timeouts
 2016-01-20 16:03:53 -08:00
Diptanu Choudhury 39b263ed7f Refactoring some comments and test names 2016-01-14 15:07:24 -08:00
Diptanu Choudhury 7060001262 Added some more comments 2016-01-14 13:47:46 -08:00
Diptanu Choudhury 4d94af74f8 Added some docs and removed a redundant method 2016-01-14 13:45:48 -08:00
Diptanu Choudhury 08d8a7d527 changed the API of the client 2016-01-14 13:35:42 -08:00
Diptanu Choudhury e77be22c43 Returning an error if the list fails 2016-01-14 11:47:05 -08:00
Diptanu Choudhury 29d264ff7c Renamed AllocFile to AllocFileInfo 2016-01-13 17:18:10 -08:00
Diptanu Choudhury e1d08eeccf Closing the file handler after finishing readat 2016-01-13 14:39:06 -08:00
Diptanu Choudhury 9d18caea46 alex: making the readat more efficient 2016-01-13 12:43:13 -08:00
Diptanu Choudhury a02735e8d9 Writing contents of buffer to writer even if there was an error 2016-01-13 11:38:43 -08:00
Diptanu Choudhury 9e5d6d7fe8 Implemeted readAt 2016-01-12 22:06:42 -08:00
Diptanu Choudhury 74af0da4cd Implemented the Stat API 2016-01-12 21:28:07 -08:00
Diptanu Choudhury a1453e6180 Added the Stat API 2016-01-12 15:25:51 -08:00
Diptanu Choudhury e3d7e693dc Added methods for listing directories inside an alloc 2016-01-12 15:03:53 -08:00
Diptanu Choudhury f6fb42835e Using cgo dependencies to look up users 2015-12-15 11:12:13 -08:00
Chris Bednarski 9292a97062 Merge branch 'user-lookup-nocgo' of https://github.com/carlosdp/nomad into b-user-lookup 2015-12-01 13:44:56 -08:00
Carlos Diaz-Padron 55e49506f0 Refactor out userLookup to helper package 
Also replaces user.Lookup in exec driver
 2015-12-01 11:59:55 -08:00
Alex Dadgar c7cd7abe22 Inject the current binary into the chroot in test mode 2015-11-24 16:30:53 -08:00
Diptanu Choudhury ad4248e087 Fixed the allocdir tests 2015-11-16 13:10:57 -08:00
Carlos Diaz-Padron ffe67d8910 Remove CGO dependency for user lookup in allocdir 
os/user's user.Lookup requires that the artifact be compiled with CGO
support enabled. This change instead reads /etc/passwd directly.

The code was acquired from docker/docker#1096
 2015-11-15 21:26:34 -08:00
Alex Dadgar 6119f42d65 Check for previous embeds at entry level not folder level 2015-11-10 18:10:02 -08:00