Run tests workflow (sqlalchemy/sqlalchemy)

The Run tests workflow from sqlalchemy/sqlalchemy, explained and optimized by Latchkey.

CI health: D - needs work

Run this on Latchkey for self-healing, caching, and up to 58% lower cost.

Grade your own workflow free or run it on Latchkey →

Source: sqlalchemy/sqlalchemy.github/workflows/run-test.yamlLicense MITView source

What it does

This is the Run tests workflow from the sqlalchemy/sqlalchemy repository, a real project running GitHub Actions. It is shown here with attribution under its MIT license.

Below, Latchkey shows a faster, safer version produced by its optimization engine.

The workflow

workflow (.yml)

name: Run tests

on:
  # run on push in main or rel_* branches excluding changes are only on doc or example folders
  push:
    branches:
      - main
      - "rel_*"
      # branches used to test the workflow
      - "workflow_test_*"
    paths-ignore:
      - "examples/**"

env:
  # global env to all steps
  TOX_WORKERS: -n4

permissions:
  contents: read

jobs:
  run-test:
    name: test-${{ matrix.python-version }}-${{ matrix.os }}-${{ matrix.architecture }}-${{ matrix.build-type }}
    runs-on: ${{ matrix.os }}
    strategy:
      # run this job using this matrix, excluding some combinations below.
      matrix:
        os:
          - "ubuntu-22.04"
          - "ubuntu-22.04-arm"
          - "windows-latest"
          - "windows-11-arm"
          - "macos-latest"
        python-version:
          - "3.10"
          - "3.11"
          - "3.12"
          - "3.13"
          - "3.14"
          - "3.14t"
        build-type:
          # builds greenlet, runs asyncio tests.  includes aiosqlite driver
          - "cext-greenlet"

          # these do not install greenlet at all and skip asyncio tests.
          # does not include aiosqlite driver
          - "cext"
          - "nocext"
        architecture:
          - x64
          - x86
          - arm64

        include:
          # autocommit tests fail on the ci for some reason
          - python-version: "pypy-3.11"
            pytest-args: "-k 'not test_autocommit_on and not test_turn_autocommit_off_via_default_iso_level and not test_autocommit_isolation_level'"
            architecture: x64
            build-type: "nocext"
            os: "ubuntu-22.04"

        exclude:

          # the threaded pythons are not stable under greenlet. Even
          # though we can run individual tests, when you run the whole suite
          # with xdist and the greenlet wrapper, the workers keep crashing
          # and getting replaced
          - build-type: "cext-greenlet"
            python-version: "3.14t"

          # linux do not have x86 / arm64 python
          - os: "ubuntu-22.04"
            architecture: x86
          - os: "ubuntu-22.04"
            architecture: arm64
          # linux-arm do not have x86 / x64 python
          - os: "ubuntu-22.04-arm"
            architecture: x86
          - os: "ubuntu-22.04-arm"
            architecture: x64
          # windows des not have arm64 python
          - os: "windows-latest"
            architecture: arm64
          # macos: latests uses arm macs. no x86/x64
          - os: "macos-latest"
            architecture: x86
          - os: "macos-latest"
            architecture: x64
          - os: "windows-11-arm"
            python-version: "3.10"
          - os: "windows-11-arm"
            architecture: x86
          - os: "windows-11-arm"
            architecture: x64
          # 3.14t is not yet supported on windows-11-arm
          - os: "windows-11-arm"
            python-version: "3.14t"

      fail-fast: false

    # steps to run in each job. Some are github actions, others run shell commands
    steps:
      - name: Checkout repo
        uses: actions/checkout@v7

      - name: Set up python
        uses: actions/setup-python@v6
        with:
          python-version: ${{ matrix.python-version }}
          architecture: ${{ matrix.architecture }}

      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install --upgrade nox setuptools
          pip list

      - name: Run tests
        run: nox -v -s github-${{ matrix.build-type }} -- ${{ matrix.pytest-args }}
        continue-on-error: ${{ matrix.python-version == 'pypy-3.11' }}

  run-nox:
    name: ${{ matrix.nox-env }}-${{ matrix.python-version }}
    runs-on: ${{ matrix.os }}
    strategy:
      # run this job using this matrix, excluding some combinations below.
      matrix:
        os:
          - "ubuntu-22.04"
        python-version:
          - "3.14"
        nox-env:
          - mypy
          - pep484
          - pep8

      fail-fast: false

    # steps to run in each job. Some are github actions, others run shell commands
    steps:
      - name: Checkout repo
        uses: actions/checkout@v7

      - name: Set up python
        uses: actions/setup-python@v6
        with:
          python-version: ${{ matrix.python-version }}
          architecture: ${{ matrix.architecture }}

      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install --upgrade nox setuptools
          pip list

      - name: Run nox
        run: nox -v -e ${{ matrix.nox-env }} ${{ matrix.pytest-args }}

The same workflow, on Latchkey

Estimated ~20% faster on cache hits, plus fewer wasted runs and a safer supply chain. Added and changed lines are highlighted.

name: Run tests
 
on:
  # run on push in main or rel_* branches excluding changes are only on doc or example folders
  push:
    branches:
      - main
      - "rel_*"
      # branches used to test the workflow
      - "workflow_test_*"
    paths-ignore:
      - "examples/**"
 
env:
  # global env to all steps
  TOX_WORKERS: -n4
 
permissions:
  contents: read
 
concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true
 
jobs:
  run-test:
    timeout-minutes: 30
    name: test-${{ matrix.python-version }}-${{ matrix.os }}-${{ matrix.architecture }}-${{ matrix.build-type }}
    runs-on: ${{ matrix.os }}
    strategy:
      # run this job using this matrix, excluding some combinations below.
      matrix:
        os:
          - "ubuntu-22.04"
          - "ubuntu-22.04-arm"
          - "windows-latest"
          - "windows-11-arm"
          - "macos-latest"
        python-version:
          - "3.10"
          - "3.11"
          - "3.12"
          - "3.13"
          - "3.14"
          - "3.14t"
        build-type:
          # builds greenlet, runs asyncio tests.  includes aiosqlite driver
          - "cext-greenlet"
 
          # these do not install greenlet at all and skip asyncio tests.
          # does not include aiosqlite driver
          - "cext"
          - "nocext"
        architecture:
          - x64
          - x86
          - arm64
 
        include:
          # autocommit tests fail on the ci for some reason
          - python-version: "pypy-3.11"
            pytest-args: "-k 'not test_autocommit_on and not test_turn_autocommit_off_via_default_iso_level and not test_autocommit_isolation_level'"
            architecture: x64
            build-type: "nocext"
            os: "ubuntu-22.04"
 
        exclude:
 
          # the threaded pythons are not stable under greenlet. Even
          # though we can run individual tests, when you run the whole suite
          # with xdist and the greenlet wrapper, the workers keep crashing
          # and getting replaced
          - build-type: "cext-greenlet"
            python-version: "3.14t"
 
          # linux do not have x86 / arm64 python
          - os: "ubuntu-22.04"
            architecture: x86
          - os: "ubuntu-22.04"
            architecture: arm64
          # linux-arm do not have x86 / x64 python
          - os: "ubuntu-22.04-arm"
            architecture: x86
          - os: "ubuntu-22.04-arm"
            architecture: x64
          # windows des not have arm64 python
          - os: "windows-latest"
            architecture: arm64
          # macos: latests uses arm macs. no x86/x64
          - os: "macos-latest"
            architecture: x86
          - os: "macos-latest"
            architecture: x64
          - os: "windows-11-arm"
            python-version: "3.10"
          - os: "windows-11-arm"
            architecture: x86
          - os: "windows-11-arm"
            architecture: x64
          # 3.14t is not yet supported on windows-11-arm
          - os: "windows-11-arm"
            python-version: "3.14t"
 
      fail-fast: false
 
    # steps to run in each job. Some are github actions, others run shell commands
    steps:
      - name: Checkout repo
        uses: actions/checkout@v7
 
      - name: Set up python
        uses: actions/setup-python@v6
        with:
          cache: 'pip'
          python-version: ${{ matrix.python-version }}
          architecture: ${{ matrix.architecture }}
 
      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install --upgrade nox setuptools
          pip list
 
      - name: Run tests
        run: nox -v -s github-${{ matrix.build-type }} -- ${{ matrix.pytest-args }}
        continue-on-error: ${{ matrix.python-version == 'pypy-3.11' }}
 
  run-nox:
    timeout-minutes: 30
    name: ${{ matrix.nox-env }}-${{ matrix.python-version }}
    runs-on: ${{ matrix.os }}
    strategy:
      # run this job using this matrix, excluding some combinations below.
      matrix:
        os:
          - "ubuntu-22.04"
        python-version:
          - "3.14"
        nox-env:
          - mypy
          - pep484
          - pep8
 
      fail-fast: false
 
    # steps to run in each job. Some are github actions, others run shell commands
    steps:
      - name: Checkout repo
        uses: actions/checkout@v7
 
      - name: Set up python
        uses: actions/setup-python@v6
        with:
          cache: 'pip'
          python-version: ${{ matrix.python-version }}
          architecture: ${{ matrix.architecture }}
 
      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install --upgrade nox setuptools
          pip list
 
      - name: Run nox
        run: nox -v -e ${{ matrix.nox-env }} ${{ matrix.pytest-args }}

What changed

Cancel superseded runs when a branch or PR gets a newer push.
Cache dependency installs on the setup step so they are served from cache.
Add a job timeout so a hung step cannot burn hours of runner time.

What Latchkey heals here

This workflow has steps that commonly fail on transient issues (network, registries, flaky browsers). On Latchkey managed runners they are detected, retried, and self-healed instead of failing your build:

Dependency installs

This workflow runs 2 jobs (273 with the matrix expanded) per trigger. On Latchkey the same minutes cost up to 58% less than GitHub-hosted, with zero queue time.

Actions used in this workflow

actions/checkout actions/setup-python