Tests workflow (ultimate-notion/ultimate-notion)

The Tests workflow from ultimate-notion/ultimate-notion, explained and optimized by Latchkey.

CI health: B - good

Point runs-on at Latchkey and get job timeouts, SHA-pinned actions, self-healing for flaky steps, and up to 58% lower cost, applied automatically.

Grade your own workflow free or run it on Latchkey →

Source: ultimate-notion/ultimate-notion.github/workflows/run-tests.ymlLicense MITView source

What it does

This is the Tests workflow from the ultimate-notion/ultimate-notion repository, a real project running GitHub Actions. It is shown here with attribution under its MIT license.

Below, Latchkey shows a faster, safer version produced by its optimization engine.

The workflow

workflow (.yml)

name: Tests

on:
  push:
    branches:
      - main
  pull_request:
    branches:
      - main

concurrency:
  group: test-${{ github.head_ref }}
  cancel-in-progress: false

env:
  PYTHONUNBUFFERED: "1"
  FORCE_COLOR: "1"
  NOTION_TOKEN: "${{ secrets.NOTION_AUTH_TOKEN }}"
  # PIP_COMPILE_DISABLE: true  # would make sense but it's a bug in https://github.com/juftin/hatch-pip-compile/pull/100
  PIPX_HOME: "${{ github.workspace }}/.pipx"

jobs:
  run-github-job-matrix:
    name: Python ${{ matrix.python-version }} on ${{ startsWith(matrix.os, 'macos-') && 'macOS' || startsWith(matrix.os, 'windows-') && 'Windows' || 'Linux' }}
    runs-on: ${{ matrix.os }}
    strategy:
      max-parallel: 0  # set to 1 to avoid time-outs and conflicts when speaking to the Notion API
      fail-fast: true # kill everything after first error to save resources
      matrix:
        include:
          # Linux - all Python versions
          - os: ubuntu-latest
            python-version: '3.10'
          - os: ubuntu-latest
            python-version: '3.11'
          - os: ubuntu-latest
            python-version: '3.12'
          - os: ubuntu-latest
            python-version: '3.13'
          - os: ubuntu-latest
            python-version: '3.14'
          # Windows - only Python 3.10
          - os: windows-latest
            python-version: '3.10'
          # macOS - only Python 3.10
          - os: macos-latest
            python-version: '3.10'

    steps:
    - name: Checkout
      uses: actions/checkout@v6
      with:
        fetch-depth: 0  # for git tags

    - name: Setup environment requirements
      uses: ./.github/actions/setup-venv-reqs

    - name: Install Python version (Linux/macOS)
      if: runner.os != 'Windows'
      shell: bash
      run: |
        hatch python install ${{ matrix.python-version }}
        # Save the path to the installed Python version to the environment for later injection
        PY="$(hatch --no-color python find ${{ matrix.python-version }} | tr -d '\n')"
        printf 'HATCH_PYTHON=%s\n' "$PY" >> "$GITHUB_ENV"
        cat "$GITHUB_ENV"

    - name: Install Python version (Windows)
      if: runner.os == 'Windows'
      shell: pwsh
      run: |
        hatch python install "${{ matrix.python-version }}"
        # Save the path to the installed Python version to the environment for later injection
        $PY = (hatch --no-color python find "${{ matrix.python-version }}").Trim()
        "HATCH_PYTHON=$PY" | Out-File -FilePath $env:GITHUB_ENV -Encoding utf8 -Append
        Get-Content $env:GITHUB_ENV

    - name: Lint
      if: matrix.python-version == '3.10' && runner.os == 'Linux'
      run: hatch run lint:all

    - name: Run tests and track code coverage
      run: hatch run ci

    # The macOS runner's Homebrew refuses to load the coveralls formula from
    # the untrusted coverallsapp/coveralls tap, which the Coveralls action
    # installs from. Trust the tap up front so the install succeeds. See #358.
    - name: Trust Coveralls Homebrew tap (macOS)
      if: runner.os == 'macOS'
      shell: bash
      run: |
        brew tap coverallsapp/coveralls
        brew trust coverallsapp/coveralls

    - name: Coveralls Parallel
      uses: coverallsapp/github-action@v2
      with:
        github-token: ${{ secrets.github_token }}
        flag-name: run-${{ join(matrix.*, '-') }}
        parallel: true
        path-to-lcov: coverage.lcov

  finish-coveralls:
      if: ${{ always() }}
      needs: run-github-job-matrix
      runs-on: ubuntu-latest
      steps:
      - name: Coveralls finished
        uses: coverallsapp/github-action@v2
        with:
          github-token: ${{ secrets.github_token }}
          parallel-finished: true
          path-to-lcov: coverage.lcov

  completion-tests:
    # Aggregate job with a stable name so it can be used as a single required
    # status check for branch protection / auto-merge. The matrix job names
    # vary per OS and Python version, so they cannot be required directly.
    needs: run-github-job-matrix
    runs-on: ubuntu-latest
    if: ${{ always() }} # Run even if one matrix job fails
    steps:
      - name: Check matrix job status
        run: |-
          if ! ${{ needs.run-github-job-matrix.result == 'success' }}; then
            echo "One or more matrix jobs failed"
            exit 1
          fi

The same workflow, on Latchkey

Removes redundant runs and caps runaway jobs. Added and changed lines are highlighted.

name: Tests
 
on:
  push:
    branches:
      - main
  pull_request:
    branches:
      - main
 
concurrency:
  group: test-${{ github.head_ref }}
  cancel-in-progress: false
 
env:
  PYTHONUNBUFFERED: "1"
  FORCE_COLOR: "1"
  NOTION_TOKEN: "${{ secrets.NOTION_AUTH_TOKEN }}"
  # PIP_COMPILE_DISABLE: true  # would make sense but it's a bug in https://github.com/juftin/hatch-pip-compile/pull/100
  PIPX_HOME: "${{ github.workspace }}/.pipx"
 
jobs:
  run-github-job-matrix:
    timeout-minutes: 30
    name: Python ${{ matrix.python-version }} on ${{ startsWith(matrix.os, 'macos-') && 'macOS' || startsWith(matrix.os, 'windows-') && 'Windows' || 'Linux' }}
    runs-on: ${{ matrix.os }}
    strategy:
      max-parallel: 0  # set to 1 to avoid time-outs and conflicts when speaking to the Notion API
      fail-fast: true # kill everything after first error to save resources
      matrix:
        include:
          # Linux - all Python versions
          - os: ubuntu-latest
            python-version: '3.10'
          - os: ubuntu-latest
            python-version: '3.11'
          - os: ubuntu-latest
            python-version: '3.12'
          - os: ubuntu-latest
            python-version: '3.13'
          - os: ubuntu-latest
            python-version: '3.14'
          # Windows - only Python 3.10
          - os: windows-latest
            python-version: '3.10'
          # macOS - only Python 3.10
          - os: macos-latest
            python-version: '3.10'
 
    steps:
    - name: Checkout
      uses: actions/checkout@v6
      with:
        fetch-depth: 0  # for git tags
 
    - name: Setup environment requirements
      uses: ./.github/actions/setup-venv-reqs
 
    - name: Install Python version (Linux/macOS)
      if: runner.os != 'Windows'
      shell: bash
      run: |
        hatch python install ${{ matrix.python-version }}
        # Save the path to the installed Python version to the environment for later injection
        PY="$(hatch --no-color python find ${{ matrix.python-version }} | tr -d '\n')"
        printf 'HATCH_PYTHON=%s\n' "$PY" >> "$GITHUB_ENV"
        cat "$GITHUB_ENV"
 
    - name: Install Python version (Windows)
      if: runner.os == 'Windows'
      shell: pwsh
      run: |
        hatch python install "${{ matrix.python-version }}"
        # Save the path to the installed Python version to the environment for later injection
        $PY = (hatch --no-color python find "${{ matrix.python-version }}").Trim()
        "HATCH_PYTHON=$PY" | Out-File -FilePath $env:GITHUB_ENV -Encoding utf8 -Append
        Get-Content $env:GITHUB_ENV
 
    - name: Lint
      if: matrix.python-version == '3.10' && runner.os == 'Linux'
      run: hatch run lint:all
 
    - name: Run tests and track code coverage
      run: hatch run ci
 
    # The macOS runner's Homebrew refuses to load the coveralls formula from
    # the untrusted coverallsapp/coveralls tap, which the Coveralls action
    # installs from. Trust the tap up front so the install succeeds. See #358.
    - name: Trust Coveralls Homebrew tap (macOS)
      if: runner.os == 'macOS'
      shell: bash
      run: |
        brew tap coverallsapp/coveralls
        brew trust coverallsapp/coveralls
 
    - name: Coveralls Parallel
      uses: coverallsapp/github-action@v2
      with:
        github-token: ${{ secrets.github_token }}
        flag-name: run-${{ join(matrix.*, '-') }}
        parallel: true
        path-to-lcov: coverage.lcov
 
  finish-coveralls:
    timeout-minutes: 30
      if: ${{ always() }}
      needs: run-github-job-matrix
      runs-on: latchkey-small
      steps:
      - name: Coveralls finished
        uses: coverallsapp/github-action@v2
        with:
          github-token: ${{ secrets.github_token }}
          parallel-finished: true
          path-to-lcov: coverage.lcov
 
  completion-tests:
    timeout-minutes: 30
    # Aggregate job with a stable name so it can be used as a single required
    # status check for branch protection / auto-merge. The matrix job names
    # vary per OS and Python version, so they cannot be required directly.
    needs: run-github-job-matrix
    runs-on: latchkey-small
    if: ${{ always() }} # Run even if one matrix job fails
    steps:
      - name: Check matrix job status
        run: |-
          if ! ${{ needs.run-github-job-matrix.result == 'success' }}; then
            echo "One or more matrix jobs failed"
            exit 1
          fi

What changed

Run on Latchkey managed runners with one line (runs-on), which apply the fixes below automatically and self-heal transient failures. This example uses latchkey-small; pick the runner size that fits the job.
Add a job timeout so a hung step cannot burn hours of runner time.

1 third-party action is referenced by a movable tag. Pin it to the commit SHA (Latchkey resolves and applies this automatically) so a repointed tag cannot change what runs.

This workflow runs 3 jobs per trigger. On Latchkey the same minutes cost up to 58% less than GitHub-hosted, with zero queue time.

Actions used in this workflow

actions/checkout