Mars Continuous Benchmark workflow (mars-project/mars)
The Mars Continuous Benchmark workflow from mars-project/mars, explained and optimized by Latchkey.
CI health: A - excellent
Point runs-on at Latchkey and get job timeouts, self-healing for flaky steps, and up to 58% lower cost, applied automatically.
What it does
This is the Mars Continuous Benchmark workflow from the mars-project/mars repository, a real project running GitHub Actions. It is shown here with attribution under its Apache-2.0 license.
Below, Latchkey shows a faster, safer version produced by its optimization engine.
The workflow
name: Mars Continuous Benchmark
on:
pull_request:
branches:
- master
- main
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
asv-benchmarks:
name: ASV Benchmarks
runs-on: ubuntu-latest
defaults:
run:
shell: bash -l {0}
concurrency:
# https://github.community/t/concurrecy-not-work-for-push/183068/7
group: ${{ github.event_name == 'push' && github.run_number || github.ref }}-asv-benchmarks
cancel-in-progress: true
steps:
- name: Checkout
uses: actions/checkout@v2
with:
fetch-depth: 0
- name: Set up conda 3.8
env:
PYTHON: 3.8
shell: bash
run: |
source ./ci/install-conda.sh
python -m pip install --upgrade pip setuptools wheel coverage;
- name: Install dependencies
id: build
run: |
source ./ci/reload-env.sh
export DEFAULT_VENV=$VIRTUAL_ENV
pip install numpy scipy cython oss2 asv
pip install -e ".[dev,extra]"
- name: Run ASV benchmarks
run: |
source ./ci/reload-env.sh
unset CI
cd benchmarks/asv_bench
git config --global user.email "mars_asv_benchmark@outlook.com"
git config --global user.name "Mars ASV Benchmark"
asv check -E existing
git remote add upstream https://github.com/mars-project/mars.git
git fetch upstream
git merge upstream/master
asv machine --yes
asv continuous -e -f 1.1 upstream/master HEAD
if: ${{ steps.build.outcome == 'success' }}
- name: Publish benchmarks artifact
uses: actions/upload-artifact@v2
with:
name: Benchmarks log
path: benchmarks/asv_bench/results
if: failure()
The same workflow, on Latchkey
Removes redundant runs and caps runaway jobs. Added and changed lines are highlighted.
name: Mars Continuous Benchmark on: pull_request: branches: - master - main concurrency: group: ${{ github.workflow }}-${{ github.ref }} cancel-in-progress: true jobs: asv-benchmarks: timeout-minutes: 30 name: ASV Benchmarks runs-on: latchkey-small defaults: run: shell: bash -l {0} concurrency: # https://github.community/t/concurrecy-not-work-for-push/183068/7 group: ${{ github.event_name == 'push' && github.run_number || github.ref }}-asv-benchmarks cancel-in-progress: true steps: - name: Checkout uses: actions/checkout@v2 with: fetch-depth: 0 - name: Set up conda 3.8 env: PYTHON: 3.8 shell: bash run: | source ./ci/install-conda.sh python -m pip install --upgrade pip setuptools wheel coverage; - name: Install dependencies id: build run: | source ./ci/reload-env.sh export DEFAULT_VENV=$VIRTUAL_ENV pip install numpy scipy cython oss2 asv pip install -e ".[dev,extra]" - name: Run ASV benchmarks run: | source ./ci/reload-env.sh unset CI cd benchmarks/asv_bench git config --global user.email "mars_asv_benchmark@outlook.com" git config --global user.name "Mars ASV Benchmark" asv check -E existing git remote add upstream https://github.com/mars-project/mars.git git fetch upstream git merge upstream/master asv machine --yes asv continuous -e -f 1.1 upstream/master HEAD if: ${{ steps.build.outcome == 'success' }} - name: Publish benchmarks artifact uses: actions/upload-artifact@v2 with: name: Benchmarks log path: benchmarks/asv_bench/results if: failure()
What changed
- Run on Latchkey managed runners with one line (
runs-on), which apply the fixes below automatically and self-heal transient failures. This example useslatchkey-small; pick the runner size that fits the job. - Add a job timeout so a hung step cannot burn hours of runner time.
What Latchkey heals here
This workflow has steps that commonly fail on transient issues (network, registries, flaky browsers). On Latchkey managed runners they are detected, retried, and self-healed instead of failing your build:
- Dependency installs
This workflow runs 1 job per trigger. On Latchkey the same minutes cost up to 58% less than GitHub-hosted, with zero queue time.