Benchmarks - Report workflow (napari/napari)
The Benchmarks - Report workflow from napari/napari, explained and optimized by Latchkey.
A
CI health: A - excellent
Point runs-on at Latchkey and get job timeouts, self-healing for flaky steps, and up to 58% lower cost, applied automatically.
What it does
This is the Benchmarks - Report workflow from the napari/napari repository, a real project running GitHub Actions. It is shown here with attribution under its BSD-3-Clause license.
Below, Latchkey shows a faster, safer version produced by its optimization engine.
The workflow
workflow (.yml)
# Report benchmark results to the PR
# We need a dual workflow to make sure the token has the needed permissions to post comments
# See https://stackoverflow.com/a/71683208 for more details
# When this workflow is triggered, it pulls the latest version of this file on
# the default branch. Changes to this file won't be reflected until after the
# PR is merged.
# https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#workflow_run
name: "Benchmarks - Report"
on:
workflow_run:
workflows: [Benchmarks, "Benchmark Trigger by Label"]
types:
- completed
permissions:
pull-requests: write
issues: write
jobs:
download:
runs-on: ubuntu-latest
if: ${{ github.event.workflow_run.conclusion != 'skipped' }}
steps:
- name: "Download artifact"
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
script: |
let allArtifacts = await github.rest.actions.listWorkflowRunArtifacts({
owner: context.repo.owner,
repo: context.repo.repo,
run_id: context.payload.workflow_run.id,
});
let artifactName = `asv-benchmark-results-${context.payload.workflow_run.id}-${context.payload.workflow_run.run_number}-${context.payload.workflow_run.run_attempt}`
console.log(`Artifact name: ${artifactName}`);
console.log(`All artifacts: ${JSON.stringify(allArtifacts.data.artifacts)}`);
let matchArtifact = allArtifacts.data.artifacts.filter((artifact) => {
return artifact.name == artifactName
})[0];
if (matchArtifact === undefined) {
throw TypeError('Build Artifact not found!');
}
let download = await github.rest.actions.downloadArtifact({
owner: context.repo.owner,
repo: context.repo.repo,
artifact_id: matchArtifact.id,
archive_format: 'zip',
});
let fs = require('fs');
fs.writeFileSync(`${process.env.GITHUB_WORKSPACE}/asv_results.zip`, Buffer.from(download.data));
- name: Unzip and prepare data
run: |
unzip asv_results.zip
# combine the Qt and non-Qt messages
cat message_Qt.txt message_non-Qt.txt > message.txt
- name: Replace URLs
run: |
sed -i "s@||BENCHMARK_CI_LOGS_URL||@${GITHUB_EVENT_WORKFLOW_RUN_HTML_URL}@g" message.txt
env:
GITHUB_EVENT_WORKFLOW_RUN_HTML_URL: ${{ github.event.workflow_run.html_url }}
- name: Collect PR number if available
run: |
if [[ -f pr_number ]]; then
echo "PR_NUMBER=$(cat pr_number)" >> "$GITHUB_ENV"
fi
- name: "Comment on PR"
if: env.PR_NUMBER != ''
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
let fs = require('fs');
let issue_number = Number(process.env.PR_NUMBER);
let body = fs.readFileSync('message.txt', 'utf8');
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: issue_number,
body: body,
});
- name: "Remove run-benchmarks label"
if: env.PR_NUMBER != ''
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
let fs = require('fs');
let issue_number = Number(process.env.PR_NUMBER);
await github.rest.issues.removeLabel({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: issue_number,
name: 'run-benchmarks',
});
The same workflow, on Latchkey
Removes redundant runs and caps runaway jobs. Added and changed lines are highlighted.
# Report benchmark results to the PR # We need a dual workflow to make sure the token has the needed permissions to post comments # See https://stackoverflow.com/a/71683208 for more details # When this workflow is triggered, it pulls the latest version of this file on # the default branch. Changes to this file won't be reflected until after the # PR is merged. # https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#workflow_run name: "Benchmarks - Report" on: workflow_run: workflows: [Benchmarks, "Benchmark Trigger by Label"] types: - completed permissions: pull-requests: write issues: write jobs: download: timeout-minutes: 30 runs-on: latchkey-small if: ${{ github.event.workflow_run.conclusion != 'skipped' }} steps: - name: "Download artifact" uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 with: script: | let allArtifacts = await github.rest.actions.listWorkflowRunArtifacts({ owner: context.repo.owner, repo: context.repo.repo, run_id: context.payload.workflow_run.id, }); let artifactName = `asv-benchmark-results-${context.payload.workflow_run.id}-${context.payload.workflow_run.run_number}-${context.payload.workflow_run.run_attempt}` console.log(`Artifact name: ${artifactName}`); console.log(`All artifacts: ${JSON.stringify(allArtifacts.data.artifacts)}`); let matchArtifact = allArtifacts.data.artifacts.filter((artifact) => { return artifact.name == artifactName })[0]; if (matchArtifact === undefined) { throw TypeError('Build Artifact not found!'); } let download = await github.rest.actions.downloadArtifact({ owner: context.repo.owner, repo: context.repo.repo, artifact_id: matchArtifact.id, archive_format: 'zip', }); let fs = require('fs'); fs.writeFileSync(`${process.env.GITHUB_WORKSPACE}/asv_results.zip`, Buffer.from(download.data)); - name: Unzip and prepare data run: | unzip asv_results.zip # combine the Qt and non-Qt messages cat message_Qt.txt message_non-Qt.txt > message.txt - name: Replace URLs run: | sed -i "s@||BENCHMARK_CI_LOGS_URL||@${GITHUB_EVENT_WORKFLOW_RUN_HTML_URL}@g" message.txt env: GITHUB_EVENT_WORKFLOW_RUN_HTML_URL: ${{ github.event.workflow_run.html_url }} - name: Collect PR number if available run: | if [[ -f pr_number ]]; then echo "PR_NUMBER=$(cat pr_number)" >> "$GITHUB_ENV" fi - name: "Comment on PR" if: env.PR_NUMBER != '' uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 with: github-token: ${{ secrets.GITHUB_TOKEN }} script: | let fs = require('fs'); let issue_number = Number(process.env.PR_NUMBER); let body = fs.readFileSync('message.txt', 'utf8'); await github.rest.issues.createComment({ owner: context.repo.owner, repo: context.repo.repo, issue_number: issue_number, body: body, }); - name: "Remove run-benchmarks label" if: env.PR_NUMBER != '' uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 with: github-token: ${{ secrets.GITHUB_TOKEN }} script: | let fs = require('fs'); let issue_number = Number(process.env.PR_NUMBER); await github.rest.issues.removeLabel({ owner: context.repo.owner, repo: context.repo.repo, issue_number: issue_number, name: 'run-benchmarks', });
What changed
- Run on Latchkey managed runners with one line (
runs-on), which apply the fixes below automatically and self-heal transient failures. This example useslatchkey-small; pick the runner size that fits the job. - Add a job timeout so a hung step cannot burn hours of runner time.
This workflow runs 1 job per trigger. On Latchkey the same minutes cost up to 58% less than GitHub-hosted, with zero queue time.