Benchmark workflow (swc-project/swc)

The Benchmark workflow from swc-project/swc, explained and optimized by Latchkey.

CI health: A - excellent

Point runs-on at Latchkey and get job timeouts, self-healing for flaky steps, and up to 58% lower cost, applied automatically.

Grade your own workflow free or run it on Latchkey →

Source: swc-project/swc.github/workflows/bench.ymlLicense Apache-2.0View source

What it does

This is the Benchmark workflow from the swc-project/swc repository, a real project running GitHub Actions. It is shown here with attribution under its Apache-2.0 license.

Below, Latchkey shows a faster, safer version produced by its optimization engine.

The workflow

workflow (.yml)

name: Benchmark

on:
  push:
    branches:
      - main
  pull_request:
    types: ["opened", "reopened", "synchronize"]

concurrency:
  # Do not block on main branch
  group: ${{ github.event_name == 'pull_request' && format('{0}-{1}', github.workflow, github.event.pull_request.number) || format('{0}-{1}-{2}', github.workflow, github.ref_name, github.run_id) }}
  cancel-in-progress: ${{ github.event_name == 'pull_request' }}

env:
  CI: 1
  CARGO_INCREMENTAL: 0
  CARGO_TERM_COLOR: "always"
  DIFF: 0
  # For faster CI
  RUST_LOG: "off"
  # https://github.com/swc-project/swc/pull/3742
  RUST_MIN_STACK: 4194304
  CARGO_PROFILE_RELEASE_LTO: false
  CARGO_PROFILE_BENCH_LTO: false

permissions: {}

jobs:
  list-crates:
    if: >-
      ${{ !contains(github.event.head_commit.message, 'chore: ') }}
    name: List crates to benchmark
    runs-on: ubuntu-22.04
    permissions:
      contents: read
    outputs:
      crates: ${{ steps.list-crates.outputs.crates }}
    steps:
      - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
        with:
          persist-credentials: false

      - name: List crates
        id: list-crates
        run: echo "crates=$(./scripts/bench/list-crates-with-bench.sh)" >> $GITHUB_OUTPUT

  benchmark-crate:
    name: Benchmark ${{ matrix.crate }}
    runs-on: ubuntu-22.04
    permissions:
      contents: read
    needs: list-crates
    strategy:
      matrix:
        crate: ${{fromJson(needs.list-crates.outputs.crates)}}
    steps:
      - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
        with:
          persist-credentials: false

      - name: Install Rust
        uses: dtolnay/rust-toolchain@29eef336d9b2848a0b548edc03f92a220660cdb8 # stable

      - uses: Swatinem/rust-cache@c19371144df3bb44fab255c43d04cbc2ab54d1c4 # v2.9.1
        # Some crates are too slow to build
        if: matrix.crate == 'swc'
        with:
          shared-key: "bench-${{ matrix.crate }}"
          key: "bench-${{ matrix.crate }}"
          cache-all-crates: true
          save-if: ${{ github.ref == 'refs/heads/main' }}

      - name: Install cargo-codspeed
        uses: taiki-e/install-action@3235f8901fd37ffed0052b276cec25a362fb82e9 # v2.77.7
        with:
          tool: cargo-codspeed@3.0.2

      - name: Build the benchmark target(s)
        run: ./scripts/bench/build-crate.sh ${{ matrix.crate }}

      - name: Run the benchmarks
        uses: CodSpeedHQ/action@76578c2a7ddd928664caa737f0e962e3085d4e7c # v3.8.1
        with:
          run: cargo codspeed run
          token: ${{ secrets.CODSPEED_TOKEN }}

The same workflow, on Latchkey

Removes redundant runs and caps runaway jobs. Added and changed lines are highlighted.

name: Benchmark
 
on:
  push:
    branches:
      - main
  pull_request:
    types: ["opened", "reopened", "synchronize"]
 
concurrency:
  # Do not block on main branch
  group: ${{ github.event_name == 'pull_request' && format('{0}-{1}', github.workflow, github.event.pull_request.number) || format('{0}-{1}-{2}', github.workflow, github.ref_name, github.run_id) }}
  cancel-in-progress: ${{ github.event_name == 'pull_request' }}
 
env:
  CI: 1
  CARGO_INCREMENTAL: 0
  CARGO_TERM_COLOR: "always"
  DIFF: 0
  # For faster CI
  RUST_LOG: "off"
  # https://github.com/swc-project/swc/pull/3742
  RUST_MIN_STACK: 4194304
  CARGO_PROFILE_RELEASE_LTO: false
  CARGO_PROFILE_BENCH_LTO: false
 
permissions: {}
 
jobs:
  list-crates:
    timeout-minutes: 30
    if: >-
      ${{ !contains(github.event.head_commit.message, 'chore: ') }}
    name: List crates to benchmark
    runs-on: latchkey-small
    permissions:
      contents: read
    outputs:
      crates: ${{ steps.list-crates.outputs.crates }}
    steps:
      - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
        with:
          persist-credentials: false
 
      - name: List crates
        id: list-crates
        run: echo "crates=$(./scripts/bench/list-crates-with-bench.sh)" >> $GITHUB_OUTPUT
 
  benchmark-crate:
    timeout-minutes: 30
    name: Benchmark ${{ matrix.crate }}
    runs-on: latchkey-small
    permissions:
      contents: read
    needs: list-crates
    strategy:
      matrix:
        crate: ${{fromJson(needs.list-crates.outputs.crates)}}
    steps:
      - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
        with:
          persist-credentials: false
 
      - name: Install Rust
        uses: dtolnay/rust-toolchain@29eef336d9b2848a0b548edc03f92a220660cdb8 # stable
 
      - uses: Swatinem/rust-cache@c19371144df3bb44fab255c43d04cbc2ab54d1c4 # v2.9.1
        # Some crates are too slow to build
        if: matrix.crate == 'swc'
        with:
          shared-key: "bench-${{ matrix.crate }}"
          key: "bench-${{ matrix.crate }}"
          cache-all-crates: true
          save-if: ${{ github.ref == 'refs/heads/main' }}
 
      - name: Install cargo-codspeed
        uses: taiki-e/install-action@3235f8901fd37ffed0052b276cec25a362fb82e9 # v2.77.7
        with:
          tool: cargo-codspeed@3.0.2
 
      - name: Build the benchmark target(s)
        run: ./scripts/bench/build-crate.sh ${{ matrix.crate }}
 
      - name: Run the benchmarks
        uses: CodSpeedHQ/action@76578c2a7ddd928664caa737f0e962e3085d4e7c # v3.8.1
        with:
          run: cargo codspeed run
          token: ${{ secrets.CODSPEED_TOKEN }}

What changed

Run on Latchkey managed runners with one line (runs-on), which apply the fixes below automatically and self-heal transient failures. This example uses latchkey-small; pick the runner size that fits the job.
Add a job timeout so a hung step cannot burn hours of runner time.

This workflow runs 2 jobs per trigger. On Latchkey the same minutes cost up to 58% less than GitHub-hosted, with zero queue time.

Actions used in this workflow

actions/checkout Swatinem/rust-cache