Commit Graph

56 Commits

Author SHA1 Message Date
hashicorp-copywrite[bot] 005636afa0 [COMPLIANCE] Add Copyright and License Headers 2023-04-10 15:36:59 +00:00
Piotr Kazmierczak 14b53df3b6
renamed stanza to block for consistency with other projects (#15941) 2023-01-30 15:48:43 +01:00
Seth Hoenig cd75858f4a
api: purge testify and pretty dependencies (#15627)
* api: swap testify for test (acl)

* api: swap testify for test (agent)

 Please enter the commit message for your changes. Lines starting

* api: swap testify for test (allocations)

* api: swap testify for test (api)

* api: swap testify for test (compose)

* api: swap testify for test (constraint)

* api: swap testify for test (consul)

* api: swap testify for test (csi)

* api: swap testify for test (evaluations)

* api: swap testify for test (event stream)

* api: swap testify for test (fs)

* api: swap testify for test (ioutil)

* api: swap testify for test (jobs)

* api: swap testify for test (keyring)

* api: swap testify for test (operator_ent)

* api: swap testify for test (operator_metrics)

* api: swap testify for test (operator)

* api: swap testify for test (quota)

* api: swap testify for test (resources)

* api: swap testify for test (fix operator_metrics)

* api: swap testify for test (scaling)

* api: swap testify for test (search)

* api: swap testify for test (sentinel)

* api: swap testify for test (services)

* api: swap testify for test (status)

* api: swap testify for test (system)

* api: swap testify for test (tasks)

* api: swap testify for test (utils)

* api: swap testify for test (variables)

* api: remove dependencies on testify and pretty
2023-01-01 12:57:26 -06:00
Piotr Kazmierczak b63944b5c1
cleanup: replace TypeToPtr helper methods with pointer.Of (#14151)
Bumping compile time requirement to go 1.18 allows us to simplify our pointer helper methods.
2022-08-17 18:26:34 +02:00
Luiz Aoqui a8cc633156
vault: revert support for entity aliases (#12723)
After a more detailed analysis of this feature, the approach taken in
PR #12449 was found to be not ideal due to poor UX (users are
responsible for setting the entity alias they would like to use) and
issues around jobs potentially masquerading itself as another Vault
entity.
2022-04-22 10:46:34 -04:00
Luiz Aoqui ab7eb5de6e
Support Vault entity aliases (#12449)
Move some common Vault API data struct decoding out of the Vault client
so it can be reused in other situations.

Make Vault job validation its own function so it's easier to expand it.

Rename the `Job.VaultPolicies` method to just `Job.Vault` since it
returns the full Vault block, not just their policies.

Set `ChangeMode` on `Vault.Canonicalize`.

Add some missing tests.

Allows specifying an entity alias that will be used by Nomad when
deriving the task Vault token.

An entity alias assigns an indentity to a token, allowing better control
and management of Vault clients since all tokens with the same indentity
alias will now be considered the same client. This helps track Nomad
activity in Vault's audit logs and better control over Vault billing.

Add support for a new Nomad server configuration to define a default
entity alias to be used when deriving Vault tokens. This default value
will be used if the task doesn't have an entity alias defined.
2022-04-05 14:18:10 -04:00
Seth Hoenig 3943dd1e16 ci: use serial testing for api in CI
This is a followup to running tests in serial in CI.
Since the API package cannot import anything outside of api/,
copy the ci.Parallel function into api/internal/testutil, and
have api tests use that.
2022-03-17 08:35:01 -05:00
Derek Strickland 0a8e03f0f7
Expose Consul template configuration parameters (#11606)
This PR exposes the following existing`consul-template` configuration options to Nomad jobspec authors in the `{job.group.task.template}` stanza.

- `wait`

It also exposes the following`consul-template` configuration to Nomad operators in the `{client.template}` stanza.

- `max_stale`
- `block_query_wait`
- `consul_retry`
- `vault_retry` 
- `wait` 

Finally, it adds the following new Nomad-specific configuration to the `{client.template}` stanza that allows Operators to set bounds on what `jobspec` authors configure.

- `wait_bounds`

Co-authored-by: Tim Gross <tgross@hashicorp.com>
Co-authored-by: Michael Schurter <mschurter@hashicorp.com>
2022-01-10 10:19:07 -05:00
Seth Hoenig f17ba33f61 consul: plubming for specifying consul namespace in job/group
This PR adds the common OSS changes for adding support for Consul Namespaces,
which is going to be a Nomad Enterprise feature. There is no new functionality
provided by this changeset and hopefully no new bugs.
2021-04-05 10:03:19 -06:00
Seth Hoenig bb8a5816a0 jobspec: add support for headers in artifact stanza
This PR adds the ability to set HTTP headers when downloading
an artifact from an `http` or `https` resource.

The implementation in `go-getter` is such that a new `HTTPGetter`
must be created for each artifact that sets headers (as opposed
to conveniently setting headers per-request). This PR maintains
the memoization of the default Getter objects, creating new ones
only for artifacts where headers are set.

Closes #9306
2020-11-13 12:03:54 -06:00
Chris Baker bdbb066ec8 fixed api tests for changes 2020-07-04 19:23:58 +00:00
Nick Ethier a87e91e971
test: fix up testing around host networks 2020-06-19 13:53:31 -04:00
Chris Baker 5979d6a81e more testing for ScalingPolicy, mainly around parsing and canonicalization for Min/Max 2020-03-24 19:43:50 +00:00
Chris Baker 233db5258a changes to Canonicalize, Validate, and api->struct conversion so that tg.Count, tg.Scaling.Min/Max are well-defined with reasonable defaults.
- tg.Count defaults to tg.Scaling.Min if present (falls back on previous default of 1 if Scaling is absent)
- Validate() enforces tg.Scaling.Min <= tg.Count <= tg.Scaling.Max

modification in ApiScalingPolicyToStructs, api.TaskGroup.Validate so that defaults are handled for TaskGroup.Count and
2020-03-24 13:57:17 +00:00
Jasmine Dahilig 1485b342e2 remove deadline code for now 2020-03-21 17:52:56 -04:00
Jasmine Dahilig b69e8e3a42 remove api package dependency on structs package 2020-03-21 17:52:55 -04:00
Jasmine Dahilig f6e58d6dad add canonicalize in the right place 2020-03-21 17:52:41 -04:00
James Rasell 6463532577
Fix panic when canonicalizing a jobspec with incorrect job type.
When canonicalizing the ReschedulePolicy a panic was possible if
the passed job type was not valid. This change protects against
this possibility, in a verbose way to ensure the code path is
clear.
2020-02-21 09:14:36 +01:00
Danielle Lancashire 4fbcc668d0
volumes: Add support for mount propagation
This commit introduces support for configuring mount propagation when
mounting volumes with the `volume_mount` stanza on Linux targets.

Similar to Kubernetes, we expose 3 options for configuring mount
propagation:

- private, which is equivalent to `rprivate` on Linux, which does not allow the
           container to see any new nested mounts after the chroot was created.

- host-to-task, which is equivalent to `rslave` on Linux, which allows new mounts
                that have been created _outside of the container_ to be visible
                inside the container after the chroot is created.

- bidirectional, which is equivalent to `rshared` on Linux, which allows both
                 the container to see new mounts created on the host, but
                 importantly _allows the container to create mounts that are
                 visible in other containers an don the host_

private and host-to-task are safe, but bidirectional mounts can be
dangerous, as if the code inside a container creates a mount, and does
not clean it up before tearing down the container, it can cause bad
things to happen inside the kernel.

To add a layer of safety here, we require that the user has ReadWrite
permissions on the volume before allowing bidirectional mounts, as a
defense in depth / validation case, although creating mounts should also require
a priviliged execution environment inside the container.
2019-10-14 14:09:58 +02:00
Tim Gross a0e923f46c add optional task field to group service checks 2019-08-20 09:35:31 -04:00
Nick Ethier 6c160df689
fix tests from introducing new struct fields 2019-07-31 01:03:16 -04:00
Lang Martin 698e9d4940 tasks_test assert merging behavior around Canonicalize 2019-07-18 13:36:06 -04:00
Lang Martin b5fd735960 add update AutoPromote bool 2019-05-22 12:32:08 -04:00
Jeff Mitchell 13dab7dd24
Divest api/ package of deps elsewhere in the nomad repo. (#5488)
* Divest api/ package of deps elsewhere in the nomad repo.

This will allow making api/ a module without then pulling in the
external repo, leading to a package name conflict.

This required some migration of tests to an apitests/ folder (can be
moved anywhere as it has no deps on it). It also required some
duplication of code, notably some test helpers from api/ -> apitests/
and part (but not all) of testutil/ -> api/testutil/.

Once there's more separation and an e.g. sdk/ folder those can be
removed in favor of a dep on the sdk/ folder, provided the sdk/ folder
doesn't depend on api/ or /.

* Also remove consul dep from api/ package

* Fix stupid linters

* Some restructuring
2019-03-29 14:47:40 -04:00
Alex Dadgar 41265d4d61 Change types of weights on spread/affinity 2019-01-30 12:20:38 -08:00
Mahmood Ali 7bdd43f3e0 api: avoid codegen for syncing
Given that the values will rarely change, specially considering that any
changes would be backward incompatible change.  As such, it's simpler to
keep syncing manually in the rare occasion and avoid the syncing code
overhead.
2019-01-18 18:52:31 -05:00
Mahmood Ali b768b31e3e api: embed pointer conversion functions
Embed pointer conversion functions in the API package to avoid
unnecessary package dependency.  `helper` package imports more
dependencies relevant for internal use (e.g. `hcl`).
2019-01-18 15:31:04 -05:00
Mahmood Ali 253532ec00 api: avoid import nomad/structs pkg
nomad/structs is an internal package and imports many libraries (e.g.
raft, codec) that are not relevant to api clients, and may cause
unnecessary dependency pain (e.g. `github.com/ugorji/go/codec`
version is very old now).

Here, we add a code generator that imports the relevant constants from
`nomad/structs`.

I considered using this approach for other structs, but didn't find a
quick viable way to reduce duplication.  `nomad/structs` use values as
struct fields (e.g. `string`), while `api` uses value pointer (e.g.
`*string`) instead.  Also, sometimes, `api` structs contain deprecated
fields or additional documentation, so simple copy-paste doesn't work.
For these reasons, I opt to keep the status quo.
2019-01-18 14:51:19 -05:00
Danielle Tomlinson 587405079d api: Validate the slash variant of a given path
This validates the slash variant of a RelativeDest, rather than the
platform native version, to support test execution on windows.
2019-01-17 18:43:13 +01:00
Preetha Appan b46728a88b
Make spread weight a pointer with default value if unset 2019-01-11 10:31:21 -06:00
Alex Dadgar 1e3c3cb287 Deprecate IOPS
IOPS have been modelled as a resource since Nomad 0.1 but has never
actually been detected and there is no plan in the short term to add
detection. This is because IOPS is a bit simplistic of a unit to define
the performance requirements from the underlying storage system. In its
current state it adds unnecessary confusion and can be removed without
impacting any users. This PR leaves IOPS defined at the jobspec parsing
level and in the api/ resources since these are the two public uses of
the field. These should be considered deprecated and only exist to allow
users to stop using them during the Nomad 0.9.x release. In the future,
there should be no expectation that the field will exist.
2018-12-06 15:09:26 -08:00
Preetha Appan 659cfa3f64
Parsing and API layer for spread stanza 2018-09-04 16:10:11 -05:00
Preetha Appan 1c0b123777
Fix test 2018-09-04 16:10:11 -05:00
Preetha Appan c407e3626f
More review comments 2018-09-04 16:10:11 -05:00
Preetha Appan 9f0caa9c3d
Affinity parsing, api and structs 2018-09-04 16:10:11 -05:00
Alex Dadgar e5caaf3358
Small test fix 2018-05-07 14:50:01 -05:00
Preetha Appan 33e170c15d
s/linear/constant/g 2018-03-26 14:45:09 -05:00
Alex Dadgar 02019f216a Correct defaulting 2018-03-21 16:51:44 -07:00
Preetha Appan 9a5e6edf1f
Rename DelayCeiling to MaxDelay 2018-03-14 16:10:32 -05:00
Preetha Appan 5f50c3d618
Add new reschedule options to API layer and unit tests 2018-03-14 16:10:32 -05:00
Preetha Appan e09ea8c0b0
Address code review comments 2018-01-31 09:58:05 -06:00
Preetha Appan 1f834d1a31
Add reschedule policy to API, and HCL parsing support. 2018-01-31 09:56:53 -06:00
Michael Schurter 224046ece8 Invert and test CheckRestart merge logic 2018-01-09 15:17:07 -08:00
Alex Dadgar 4173834231 Enable more linters 2017-09-26 15:26:33 -07:00
Alex Dadgar 698387e103 Don't merge empty update from job into task groups
This PR fixes an issue in which we would merge an update stanza from the
job into the task groups even if it was empty. This broke round
tripping a job (reading from the api and resubmitting as JSON)

Fixes https://github.com/hashicorp/nomad/issues/3132
2017-08-30 11:35:19 -07:00
Alex Dadgar 789751d84e Parallel 2017-07-21 16:33:04 -07:00
Michael Schurter 0d3bdf7210 Add support for go-getter modes
Fixes #2678
2017-07-06 10:45:44 -07:00
Diptanu Choudhury d0d7c92bab Added tests 2017-02-16 13:52:39 -08:00
Diptanu Choudhury 7567209857 Making the job spec return api.Job 2017-02-16 13:52:39 -08:00
Alex Dadgar 156d5845d1 Fix api test 2015-12-18 12:17:54 -08:00