b67bd4c377
* Sample percy test added * Node engine up to 14.x for UI prep * Force ui test rerun * Updated config.yml * Node v upgraded to 14 for docker image * Expect length in test * Running ember tests under percy exec * Percy exec format * Percy cli added * Noop to rerun tests with updated percy_token * Evals full list and details open snapshots * Pretty legit use of assert so disable the warning * Jobs list tests * Snapshots for top-level clients, servers, ACL, topology, and storage lists * Expect caveat for Topology test * Stabilizing tests with faker seeded to 1 * Seed-stabilizing any tests with percySnapshots * Faker import * Drop unused param * Assets and test audit using an older node version * New strategy: avoid seeding, just use percyCSS to hide certain things
84 lines
2.6 KiB
YAML
84 lines
2.6 KiB
YAML
name: Ember test audit comparison
|
||
on:
|
||
pull_request:
|
||
paths:
|
||
- 'ui/**'
|
||
|
||
defaults:
|
||
run:
|
||
working-directory: ui
|
||
|
||
# There’s currently no way to share steps between jobs so there’s a lot of duplication
|
||
# for running the audit for the base and PR.
|
||
jobs:
|
||
time-base:
|
||
runs-on: ubuntu-latest
|
||
steps:
|
||
- uses: actions/checkout@v2
|
||
with:
|
||
ref: ${{ github.event.pull_request.base.sha }}
|
||
- uses: nanasess/setup-chromedriver@master
|
||
- name: Use Node.js
|
||
uses: actions/setup-node@v2
|
||
with:
|
||
node-version: '14'
|
||
- run: yarn --frozen-lockfile
|
||
- run: mkdir -p /tmp/test-reports
|
||
- run: npx ember-test-audit 3 --json --output ../base-audit.json
|
||
- name: Upload result
|
||
uses: actions/upload-artifact@v2
|
||
with:
|
||
name: base-audit
|
||
path: base-audit.json
|
||
time-pr:
|
||
runs-on: ubuntu-latest
|
||
steps:
|
||
- uses: actions/checkout@v2
|
||
- uses: nanasess/setup-chromedriver@master
|
||
- name: Use Node.js
|
||
uses: actions/setup-node@v2
|
||
with:
|
||
node-version: '14'
|
||
- run: yarn --frozen-lockfile
|
||
- run: mkdir -p /tmp/test-reports
|
||
- run: npx ember-test-audit 3 --json --output ../pr-audit.json
|
||
- name: Upload result
|
||
uses: actions/upload-artifact@v2
|
||
with:
|
||
name: pr-audit
|
||
path: pr-audit.json
|
||
compare:
|
||
needs: [time-base, time-pr]
|
||
runs-on: ubuntu-latest
|
||
steps:
|
||
- uses: actions/download-artifact@v2
|
||
with:
|
||
name: base-audit
|
||
- uses: actions/download-artifact@v2
|
||
with:
|
||
name: pr-audit
|
||
- uses: backspace/ember-test-audit-comparison-action@v2
|
||
with:
|
||
base-report-path: base-audit.json
|
||
comparison-report-path: pr-audit.json
|
||
base-identifier: ${{ github.event.pull_request.base.ref }}
|
||
comparison-identifier: ${{ github.event.pull_request.head.sha }}
|
||
timing-output-path: audit-diff.md
|
||
flakiness-output-path: flakiness-report.md
|
||
- uses: marocchino/sticky-pull-request-comment@v2.0.0
|
||
with:
|
||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||
path: audit-diff.md
|
||
- name: Check for existence of flakiness report
|
||
id: check_file
|
||
uses: andstor/file-existence-action@v1
|
||
with:
|
||
files: "flakiness-report.md"
|
||
- name: comment PR
|
||
if: steps.check_file.outputs.files_exists == 'true'
|
||
uses: machine-learning-apps/pr-comment@1.0.0
|
||
env:
|
||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||
with:
|
||
path: flakiness-report.md
|