Skip to content
Latchkey

Mars Continuous Benchmark workflow (mars-project/mars)

The Mars Continuous Benchmark workflow from mars-project/mars, explained and optimized by Latchkey.

A

CI health: A - excellent

Point runs-on at Latchkey and get job timeouts, self-healing for flaky steps, and up to 58% lower cost, applied automatically.

Grade your own workflow free or run it on Latchkey →
Source: mars-project/mars.github/workflows/benchmark-ci.ymlLicense Apache-2.0View source

What it does

This is the Mars Continuous Benchmark workflow from the mars-project/mars repository, a real project running GitHub Actions. It is shown here with attribution under its Apache-2.0 license.

Below, Latchkey shows a faster, safer version produced by its optimization engine.

The workflow

workflow (.yml)
name: Mars Continuous Benchmark

on:
  pull_request:
    branches:
      - master
      - main

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

jobs:
  asv-benchmarks:
    name: ASV Benchmarks
    runs-on: ubuntu-latest
    defaults:
      run:
        shell: bash -l {0}

    concurrency:
      # https://github.community/t/concurrecy-not-work-for-push/183068/7
      group: ${{ github.event_name == 'push' && github.run_number || github.ref }}-asv-benchmarks
      cancel-in-progress: true

    steps:
      - name: Checkout
        uses: actions/checkout@v2
        with:
          fetch-depth: 0

      - name: Set up conda 3.8
        env:
          PYTHON: 3.8
        shell: bash
        run: |
          source ./ci/install-conda.sh
          python -m pip install --upgrade pip setuptools wheel coverage;

      - name: Install dependencies
        id: build
        run: |
          source ./ci/reload-env.sh
          export DEFAULT_VENV=$VIRTUAL_ENV
          pip install numpy scipy cython oss2 asv
          pip install -e ".[dev,extra]"

      - name: Run ASV benchmarks
        run: |
          source ./ci/reload-env.sh
          unset CI
          cd benchmarks/asv_bench
          git config --global user.email "mars_asv_benchmark@outlook.com"
          git config --global user.name "Mars ASV Benchmark"
          asv check -E existing
          git remote add upstream https://github.com/mars-project/mars.git
          git fetch upstream
          git merge upstream/master
          asv machine --yes
          asv continuous -e -f 1.1 upstream/master HEAD
        if: ${{ steps.build.outcome == 'success' }}

      - name: Publish benchmarks artifact
        uses: actions/upload-artifact@v2
        with:
          name: Benchmarks log
          path: benchmarks/asv_bench/results
        if: failure()

The same workflow, on Latchkey

Removes redundant runs and caps runaway jobs. Added and changed lines are highlighted.

name: Mars Continuous Benchmark
 
on:
  pull_request:
    branches:
      - master
      - main
 
concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true
 
jobs:
  asv-benchmarks:
    timeout-minutes: 30
    name: ASV Benchmarks
    runs-on: latchkey-small
    defaults:
      run:
        shell: bash -l {0}
 
    concurrency:
      # https://github.community/t/concurrecy-not-work-for-push/183068/7
      group: ${{ github.event_name == 'push' && github.run_number || github.ref }}-asv-benchmarks
      cancel-in-progress: true
 
    steps:
      - name: Checkout
        uses: actions/checkout@v2
        with:
          fetch-depth: 0
 
      - name: Set up conda 3.8
        env:
          PYTHON: 3.8
        shell: bash
        run: |
          source ./ci/install-conda.sh
          python -m pip install --upgrade pip setuptools wheel coverage;
 
      - name: Install dependencies
        id: build
        run: |
          source ./ci/reload-env.sh
          export DEFAULT_VENV=$VIRTUAL_ENV
          pip install numpy scipy cython oss2 asv
          pip install -e ".[dev,extra]"
 
      - name: Run ASV benchmarks
        run: |
          source ./ci/reload-env.sh
          unset CI
          cd benchmarks/asv_bench
          git config --global user.email "mars_asv_benchmark@outlook.com"
          git config --global user.name "Mars ASV Benchmark"
          asv check -E existing
          git remote add upstream https://github.com/mars-project/mars.git
          git fetch upstream
          git merge upstream/master
          asv machine --yes
          asv continuous -e -f 1.1 upstream/master HEAD
        if: ${{ steps.build.outcome == 'success' }}
 
      - name: Publish benchmarks artifact
        uses: actions/upload-artifact@v2
        with:
          name: Benchmarks log
          path: benchmarks/asv_bench/results
        if: failure()
 

What changed

What Latchkey heals here

This workflow has steps that commonly fail on transient issues (network, registries, flaky browsers). On Latchkey managed runners they are detected, retried, and self-healed instead of failing your build:

This workflow runs 1 job per trigger. On Latchkey the same minutes cost up to 58% less than GitHub-hosted, with zero queue time.

Actions used in this workflow