luxolus
/
open-nomad
2
0
Fork 0
Code Issues 1 Pull Requests Packages Releases Wiki Activity
open-nomad/contributing/testing.md

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

40 lines
1.4 KiB
Markdown
Raw Normal View History

ci: swap ci parallelization for unconstrained gomaxprocs
2022-03-15 12:42:43 +00:00
# Writing Tests
The Nomad repository strives to maintain comprehensive unit test coverage. Any new
features, bug fixes, or refactoring should include additional or updated test cases
demonstrating correct functionality.
Each unit test should meet a few criteria:
ci: use groups of tests in gha (#15018) * [no ci] use json for grouping packages for testing * [no ci] able to get packages in group * [no ci] able to run groups of tests * [no ci] more * [no ci] try disable circle unit tests * ci: use actions/checkout@v3 * ci: rename to quick * ci: need make dev in mods cache step * ci: make compile step depend on checks step * ci: bump consul and vault versions * ci: need make dev for group tests * ci: update ci unit testing docs * docs: spell plumbing correctly Co-authored-by: Tim Gross <tgross@hashicorp.com> Co-authored-by: Tim Gross <tgross@hashicorp.com>
2022-10-27 14:02:58 +00:00
- Use [github.com/shoenig/test](https://github.com/shoenig/test)
- Prefer using `must.*`` functions
- Use `test.*`` functions when cleanup must happen, etc
- Feel free to refactor testify tests; but consider separate commits / PRs
ci: swap ci parallelization for unconstrained gomaxprocs
2022-03-15 12:42:43 +00:00
- Undo any changes to the environment
ci: use groups of tests in gha (#15018) * [no ci] use json for grouping packages for testing * [no ci] able to get packages in group * [no ci] able to run groups of tests * [no ci] more * [no ci] try disable circle unit tests * ci: use actions/checkout@v3 * ci: rename to quick * ci: need make dev in mods cache step * ci: make compile step depend on checks step * ci: bump consul and vault versions * ci: need make dev for group tests * ci: update ci unit testing docs * docs: spell plumbing correctly Co-authored-by: Tim Gross <tgross@hashicorp.com> Co-authored-by: Tim Gross <tgross@hashicorp.com>
2022-10-27 14:02:58 +00:00
- Set environment variables must be unset (use `t.Setenv`)
- Scratch files/dirs must be removed (use `t.TempDir`)
ci: swap ci parallelization for unconstrained gomaxprocs
2022-03-15 12:42:43 +00:00
- Able to run in parallel
ci: use groups of tests in gha (#15018) * [no ci] use json for grouping packages for testing * [no ci] able to get packages in group * [no ci] able to run groups of tests * [no ci] more * [no ci] try disable circle unit tests * ci: use actions/checkout@v3 * ci: rename to quick * ci: need make dev in mods cache step * ci: make compile step depend on checks step * ci: bump consul and vault versions * ci: need make dev for group tests * ci: update ci unit testing docs * docs: spell plumbing correctly Co-authored-by: Tim Gross <tgross@hashicorp.com> Co-authored-by: Tim Gross <tgross@hashicorp.com>
2022-10-27 14:02:58 +00:00
- All package level `Test*` functions should start with `ci.Parallel`
ci: swap ci parallelization for unconstrained gomaxprocs
2022-03-15 12:42:43 +00:00
- Always use dynamic scratch dirs, files
ci: swap freeport for portal in packages (#15661)
2023-01-03 17:25:20 +00:00
- Always get ports via `ci.PortAllocator.Grab()`
ci: swap ci parallelization for unconstrained gomaxprocs
2022-03-15 12:42:43 +00:00
- Log control
ci: use groups of tests in gha (#15018) * [no ci] use json for grouping packages for testing * [no ci] able to get packages in group * [no ci] able to run groups of tests * [no ci] more * [no ci] try disable circle unit tests * ci: use actions/checkout@v3 * ci: rename to quick * ci: need make dev in mods cache step * ci: make compile step depend on checks step * ci: bump consul and vault versions * ci: need make dev for group tests * ci: update ci unit testing docs * docs: spell plumbing correctly Co-authored-by: Tim Gross <tgross@hashicorp.com> Co-authored-by: Tim Gross <tgross@hashicorp.com>
2022-10-27 14:02:58 +00:00
- Logging must go through the `testing.T` (use `helper/testlog.HCLogger`)
docs: API package tests need a binary with your changes (#13029) Add a note to the contributing guide pointing out that if you're writing `api` package tests, you need to build a binary with any of your changes.
2022-05-16 15:12:54 +00:00
- Avoid excessive logging in test cases - prefer failure messages
ci: use groups of tests in gha (#15018) * [no ci] use json for grouping packages for testing * [no ci] able to get packages in group * [no ci] able to run groups of tests * [no ci] more * [no ci] try disable circle unit tests * ci: use actions/checkout@v3 * ci: rename to quick * ci: need make dev in mods cache step * ci: make compile step depend on checks step * ci: bump consul and vault versions * ci: need make dev for group tests * ci: update ci unit testing docs * docs: spell plumbing correctly Co-authored-by: Tim Gross <tgross@hashicorp.com> Co-authored-by: Tim Gross <tgross@hashicorp.com>
2022-10-27 14:02:58 +00:00
- Annotate failures with `must.Sprint\f` post-scripts
docs: API package tests need a binary with your changes (#13029) Add a note to the contributing guide pointing out that if you're writing `api` package tests, you need to build a binary with any of your changes.
2022-05-16 15:12:54 +00:00
## API tests
Testing in the `api` package requires an already-built Nomad
binary. If you're writing `api` tests, you'll need to build a Nomad
binary (ex. with `make dev`) that includes any changes your API
exercises.
ci: use groups of tests in gha (#15018) * [no ci] use json for grouping packages for testing * [no ci] able to get packages in group * [no ci] able to run groups of tests * [no ci] more * [no ci] try disable circle unit tests * ci: use actions/checkout@v3 * ci: rename to quick * ci: need make dev in mods cache step * ci: make compile step depend on checks step * ci: bump consul and vault versions * ci: need make dev for group tests * ci: update ci unit testing docs * docs: spell plumbing correctly Co-authored-by: Tim Gross <tgross@hashicorp.com> Co-authored-by: Tim Gross <tgross@hashicorp.com>
2022-10-27 14:02:58 +00:00
# CI Plumbing
See [ci/README.md] for details on how the [Core CI Tests](https://github.com/hashicorp/nomad/actions/workflows/test-core.yaml)
Github Action works.