Commit Graph

118 Commits

Author SHA1 Message Date
Alex Dadgar 3d69961c3a Must be root for TestAllocDir_CreateDir 2017-08-16 10:46:14 -07:00
Michael Schurter 46b7fd45d7 spelling 2017-08-14 16:55:59 -07:00
Michael Schurter e41a654917 switch from alloc blocker to new interface
interface has 3 implementations:

1. local for blocking and moving data locally
2. remote for blocking and moving data from another node
3. noop for allocs that don't need to block
2017-08-11 16:21:35 -07:00
Luke Farnell f0ced87b95 fixed all spelling mistakes for goreport 2017-08-07 17:13:05 -04:00
Alex Dadgar d760e68774 darwin test fixes 2017-07-22 19:48:47 -07:00
Michael Schurter 420be86e39 Test AllocDir.Copy 2017-07-17 15:46:54 -07:00
Michael Schurter 51515cbe0c Ensure allocDir is never nil and persisted safely
Fixes #2834
2017-07-17 15:46:54 -07:00
Mark Mickan c196d320f8 Add tests for migrating symlinks in alloc and local directories 2017-06-04 15:56:22 +09:30
Mark Mickan 236f24c9a4 Include symlinks in snapshots when migrating disks
Fixes #2685
2017-06-04 00:36:18 +09:30
Michael Schurter e204a287ed Refactor Consul Syncer into new ServiceClient
Fixes #2478 #2474 #1995 #2294

The new client only handles agent and task service advertisement. Server
discovery is mostly unchanged.

The Nomad client agent now handles all Consul operations instead of the
executor handling task related operations. When upgrading from an
earlier version of Nomad existing executors will be told to deregister
from Consul so that the Nomad agent can re-register the task's services
and checks.

Drivers - other than qemu - now support an Exec method for executing
abritrary commands in a task's environment. This is used to implement
script checks.

Interfaces are used extensively to avoid interacting with Consul in
tests that don't assert any Consul related behavior.
2017-04-19 12:42:47 -07:00
Michael Schurter 4b18f916d9 Set ownership on directories in chroot
Also support getOwner on all Unixes as they all have `Stat_t.{U,G}id`
2017-04-17 12:41:33 -07:00
Michael Schurter 83fb374d0b Don't disable hardlinking! 2017-04-17 11:03:15 -07:00
Michael Schurter 547d5a81b4 Chown files when copying into chroot
Fixes #2552

Not needed when hardlinking. Only adds Linux support but other OS's may
be easy.
2017-04-17 11:03:15 -07:00
Alex Dadgar f2ba6db607 Add sticky bit to temp directory
Fixes an issue where the Ruby runtime expects the sticky bit to be set
on the temp directory. The sticky bit is commonly set on the temp
directory since it is usually shared by many users. This change brings
ours in line with that assumption.
2017-04-04 10:48:29 -07:00
Alex Dadgar 3f68aae7ab TODO 2017-03-11 13:56:19 -08:00
James Nugent b3d1ef0af6 client/allocdir: Add missing functions on Solaris
This commit adds Solaris versions of the following functions:

- `linkDir`
- `unlinkDir`
- `createSecretDir`
- `removeSecretDir`

I believe this requires Go 1.8 in order to compile, as the unlink
syscall was previously missing.
2017-03-09 13:49:14 -05:00
Alex Dadgar 36f3788cb7 Fix lint errors 2017-03-02 21:03:05 -08:00
Alex Dadgar 4cba170e4b Mkdir -> MkdirAll to avoid error when folder already exists 2017-03-02 19:35:31 -08:00
Michael Schurter d12949fbbd Cleanup comments/names 2017-03-02 15:44:52 -08:00
Michael Schurter e03f64ea6a Safely ensure {dev,proc,alloc} are mounted
If they're unmounted by a reboot they'll be properly remounted.
2017-03-02 13:21:34 -08:00
Michael Schurter d5b7c0c302 unlinkDir should not error if already unlinked 2017-03-02 13:20:47 -08:00
Michael Schurter a9f9c485b4 Fix typos 2017-03-02 13:20:05 -08:00
Michael Schurter 1cfe9f88b8 Make sure unmounting the secrets dir is idemptoent 2017-03-01 17:19:39 -08:00
Michael Schurter e5a29b82db Fix allocdir Move test and make code more defensive
A change in the behavior of `os.Rename` in Go 1.8 brought to light a
difference in the logic between `{Alloc,Task}Runner` and this test:

AllocRunner builds the alloc dir, moves dirs if necessary, and then lets
TaskRunner call TaskDir.Build().

This test called `TaskDir.Build` *before* `AllocDir.Move`, so in Go 1.8
it failed to `os.Rename over` the empty {data,local} dirs.

I updated the test to behave like the real code, but I defensively added
`os.Remove` calls as a subtle change in call order shouldn't break this
code. `os.Remove` won't remove a non-empty directory, so it's still
safe.
2017-02-21 17:22:10 -08:00
Michael Schurter d8b4372904 Don't link shared alloc dir into task dir for raw_exec
Fixes running raw_exec tasks when nomad isn't root.
2017-01-18 11:28:34 -08:00
Michael Schurter c90cd0d874 Stop trying to use mount for image based drivers
Fixes #2178 and allows using Docker and other image based drivers even
when nomad is run as a non-root user.

`client/allocdir` tests can be run as a non-root user to ensure this
behavior and tests that rely on root or non-root users properly detect
their effective user and skip instead of fail.
2017-01-13 13:04:12 -08:00
Michael Schurter 86fcf96f72 Put a logger in AllocDir/TaskDir 2017-01-05 16:31:56 -08:00
Michael Schurter f43d3f074a Add comments to TaskDir 2017-01-05 16:31:55 -08:00
Michael Schurter 3ea09ba16a Move chroot building into TaskRunner
* Refactor AllocDir to have a TaskDir struct per task.
* Drivers expose filesystem isolation preference
* Fix lxc mounting of `secrets/`
2017-01-05 16:31:49 -08:00
Alex Dadgar 072ff1c3ee ensure file doesn't escape 2016-12-18 15:48:30 -08:00
Diptanu Choudhury d9f8e3a75a Fixed comments 2016-11-08 12:55:15 -08:00
Diptanu Choudhury 2132fbb68a Fixed permission issues on client 2016-11-08 10:57:29 -08:00
Alex Dadgar 5559300372 Change folder permission event when not root 2016-10-28 16:52:38 -07:00
Diptanu Choudhury 1098dc4aa3 Fixed alloc dir move tests 2016-10-26 15:17:57 -07:00
Alex Dadgar 4ae735c8ba Disallow fs to read secret directory 2016-10-24 11:14:05 -07:00
Michael Schurter 285e80ac0f Remove disk usage enforcement
Many thanks to @iverberk for the original PR (#1609), but we ended up
not wanting to ship this implementation with 0.5.

We'll come back to it after 0.5 and hopefully find a way to leverage
filesystem accounting and quotas, so we can skip the expensive polling.
2016-10-21 13:55:51 -07:00
Ben Barnard 83f647ed84 Replace "the the" with "the" in documentation and comments 2016-10-11 15:31:40 -04:00
Alex Dadgar d2837dec44 Do not allow path to escape the alloc dir for the FS commands 2016-10-03 14:58:44 -07:00
Diptanu Choudhury d50c395421 Getting snapshot of allocation from remote node (#1741)
* Added the alloc dir move

* Moving allocdirs when starting allocations

* Added the migrate flag to ephemeral disk

* Stopping migration if the allocation doesn't need migration any more

* Added the GetAllocDir method

* refactored code

* Added a test for alloc runner

* Incorporated review comments
2016-10-03 09:59:57 -07:00
Diptanu Choudhury 2b1d214b0d Avoiding copying files if they are already present in chrootw (#1753) 2016-09-27 11:43:27 -07:00
Diptanu Choudhury 12c7873db2 Closing files when files are removed 2016-09-23 22:17:53 -07:00
Diptanu Choudhury 589356fd55 Adding a snapshot endpoint on the client (#1730) 2016-09-21 21:28:12 -07:00
Alex Dadgar 6702a29071 Vault token threaded 2016-09-14 13:30:01 -07:00
Alex Dadgar eecef73302 syscall error 2016-09-02 15:00:46 -07:00
Alex Dadgar eef786dd9d Secret dir materialized in alloc/task directory 2016-09-02 12:44:05 -07:00
Alex Dadgar 2c8dd8bbd3 Revert "Introduce a Secret/ directory" 2016-09-01 17:23:15 -07:00
Alex Dadgar 9fa23e3536 Symlink on windows 2016-08-31 21:41:44 -07:00
Alex Dadgar 5d3b47e648 Address comments and reserve 2016-08-31 18:11:02 -07:00
Alex Dadgar 0626eb9619 environment variables 2016-08-31 13:56:11 -07:00
Alex Dadgar d59e14eed4 Interface + tests 2016-08-30 21:40:32 -07:00