Skip to content
Latchkey

Run Tests workflow (mottosso/Qt.py)

The Run Tests workflow from mottosso/Qt.py, explained and optimized by Latchkey.

D

CI health: D - needs work

Point runs-on at Latchkey and get caching, run de-duplication, job timeouts, self-healing for flaky steps, and up to 58% lower cost, applied automatically.

Grade your own workflow free or run it on Latchkey →
Source: mottosso/Qt.py.github/workflows/run-tox.ymlLicense MITView source

What it does

This is the Run Tests workflow from the mottosso/Qt.py repository, a real project running GitHub Actions. It is shown here with attribution under its MIT license.

Below, Latchkey shows a faster, safer version produced by its optimization engine.

The workflow

workflow (.yml)
name: Run Tests

on:
  push:
    branches:
      - "*"
  pull_request:
    branches:
      - "*"

jobs:
  membership:
    # Install a specific combination of PySide/PyQt and python. Gather the
    # available module.class member information and generate common membership.
    runs-on: ubuntu-22.04
    env:
      QT_QPA_PLATFORM: minimal
      QT_VERBOSE: 1

    steps:
      - name: Checkout code
        uses: actions/checkout@v6

      - name: Setup tox
        uses: ./.github/actions/setup-tox

      - name: Run Tox
        # Gather membership data for each each Qt binding. This uses only one
        # version of python to reduce the complexity of the output.
        # membership-end combines binding/py data into common membership report
        run: |
          which tox
          tox -e membership-py37-PySide5.13,membership-py37-PyQt5.13,membership-py39-PySide5.15,membership-py39-PyQt5.15,membership-py311-PySide6.5,membership-py311-PyQt6.7,membership-end
          ls -lha .members

      - name: Upload membership report
        uses: actions/upload-artifact@v7
        with:
          name: membership-report
          path: |
            .members/
          include-hidden-files: true

  lint:
    runs-on: ubuntu-22.04
    steps:
      - name: Checkout code
        uses: actions/checkout@v6

      - name: Setup tox
        uses: ./.github/actions/setup-tox

      - name: Run Tox
        run: |
          which tox
          tox -e check

  format:
    runs-on: ubuntu-22.04
    steps:
      - name: Checkout code
        uses: actions/checkout@v6

      - name: Setup tox
        uses: ./.github/actions/setup-tox

      - name: Run Tox
        run: |
          which tox
          tox -e format

  stubs:
    runs-on: ubuntu-22.04
    steps:
      - name: Checkout code
        uses: actions/checkout@v6

      - name: Setup tox
        uses: ./.github/actions/setup-tox

      - name: Run Tox
        run: |
          which tox
          tox -e mypy

  test:
    runs-on: ubuntu-22.04
    needs: membership
    strategy:
      matrix:
        test_env: [
          # Cy2020
          'test-py37-PySide5.13',
          'test-py37-PyQt5.13',
          # Cy2021
          'test-py37-PySide5.15',
          'test-py37-PyQt5.15',
          # Cy2022
          'test-py39-PySide5.15',
          'test-py39-PyQt5.15',
          # Cy2023
          'test-py310-PySide5.15',
          'test-py310-PyQt5.15',
          # Cy2024, Cy2025
          'test-py311-PySide6.5',
          'test-py311-PyQt6.5',
          # Cy2026
          'test-py313-PySide6.8',
          'test-py313-PyQt6.8',
          # Test newer versions of python/Qt than VFX Reference Platform has chosen
          'test-py312-PySide6.8',
          'test-py312-PyQt6.8',
          'test-py314-PySide6.9',
          'test-py314-PyQt6.9',
        ]

    env:
      QT_QPA_PLATFORM: minimal
      QT_VERBOSE: 1

    steps:
      - name: Checkout code
        uses: actions/checkout@v6

      - name: Setup tox
        uses: ./.github/actions/setup-tox

      - name: Download coverage artifacts
        # This is needed by the test_membership test
        uses: actions/download-artifact@v8
        with:
          name: membership-report
          path: .members
          merge-multiple: true
      - name: Display structure of downloaded files
        run: ls -lha .members

      - name: Tox Test - Implementation
        # Note: `--skip-missing-interpreters` prevents false success if python
        # version is not installed when testing.
        run: |
          which tox
          tox --skip-missing-interpreters false -e test-begin,${{ matrix.test_env }}-impl

      - name: Tox Test - Caveats
        run: |
          tox --skip-missing-interpreters false -e ${{ matrix.test_env }}-caveats

      - name: Tox Test - Examples
        run: |
          tox --skip-missing-interpreters false -e ${{ matrix.test_env }}-examples

  pip-package:
    # Build and upload pip packages as artifacts so we can inspect them and
    # ensure they are correct for actual release
    runs-on: ubuntu-latest

    steps:
      - name: Checkout repository
        uses: actions/checkout@v6
      - name: Set up Python
        uses: actions/setup-python@v6
        with:
          python-version: "3.10.x"
      - name: Install build dependency
        run: python3 -m pip install --upgrade build
      - name: Build a binary wheel and a source tarball
        run: python3 -m build
      - name: Upload packages.
        uses: actions/upload-artifact@v7
        with:
          name: pip-packages
          path: |
            dist/qt[._]py-*.whl
            dist/qt[._]py-*.tar.gz

The same workflow, on Latchkey

Estimated ~20% faster on cache hits, plus fewer wasted runs and a safer supply chain. Added and changed lines are highlighted.

name: Run Tests
 
on:
  push:
    branches:
      - "*"
  pull_request:
    branches:
      - "*"
 
concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true
 
jobs:
  membership:
    timeout-minutes: 30
    # Install a specific combination of PySide/PyQt and python. Gather the
    # available module.class member information and generate common membership.
    runs-on: latchkey-small
    env:
      QT_QPA_PLATFORM: minimal
      QT_VERBOSE: 1
 
    steps:
      - name: Checkout code
        uses: actions/checkout@v6
 
      - name: Setup tox
        uses: ./.github/actions/setup-tox
 
      - name: Run Tox
        # Gather membership data for each each Qt binding. This uses only one
        # version of python to reduce the complexity of the output.
        # membership-end combines binding/py data into common membership report
        run: |
          which tox
          tox -e membership-py37-PySide5.13,membership-py37-PyQt5.13,membership-py39-PySide5.15,membership-py39-PyQt5.15,membership-py311-PySide6.5,membership-py311-PyQt6.7,membership-end
          ls -lha .members
 
      - name: Upload membership report
        uses: actions/upload-artifact@v7
        with:
          name: membership-report
          path: |
            .members/
          include-hidden-files: true
 
  lint:
    timeout-minutes: 30
    runs-on: latchkey-small
    steps:
      - name: Checkout code
        uses: actions/checkout@v6
 
      - name: Setup tox
        uses: ./.github/actions/setup-tox
 
      - name: Run Tox
        run: |
          which tox
          tox -e check
 
  format:
    timeout-minutes: 30
    runs-on: latchkey-small
    steps:
      - name: Checkout code
        uses: actions/checkout@v6
 
      - name: Setup tox
        uses: ./.github/actions/setup-tox
 
      - name: Run Tox
        run: |
          which tox
          tox -e format
 
  stubs:
    timeout-minutes: 30
    runs-on: latchkey-small
    steps:
      - name: Checkout code
        uses: actions/checkout@v6
 
      - name: Setup tox
        uses: ./.github/actions/setup-tox
 
      - name: Run Tox
        run: |
          which tox
          tox -e mypy
 
  test:
    timeout-minutes: 30
    runs-on: latchkey-small
    needs: membership
    strategy:
      matrix:
        test_env: [
          # Cy2020
          'test-py37-PySide5.13',
          'test-py37-PyQt5.13',
          # Cy2021
          'test-py37-PySide5.15',
          'test-py37-PyQt5.15',
          # Cy2022
          'test-py39-PySide5.15',
          'test-py39-PyQt5.15',
          # Cy2023
          'test-py310-PySide5.15',
          'test-py310-PyQt5.15',
          # Cy2024, Cy2025
          'test-py311-PySide6.5',
          'test-py311-PyQt6.5',
          # Cy2026
          'test-py313-PySide6.8',
          'test-py313-PyQt6.8',
          # Test newer versions of python/Qt than VFX Reference Platform has chosen
          'test-py312-PySide6.8',
          'test-py312-PyQt6.8',
          'test-py314-PySide6.9',
          'test-py314-PyQt6.9',
        ]
 
    env:
      QT_QPA_PLATFORM: minimal
      QT_VERBOSE: 1
 
    steps:
      - name: Checkout code
        uses: actions/checkout@v6
 
      - name: Setup tox
        uses: ./.github/actions/setup-tox
 
      - name: Download coverage artifacts
        # This is needed by the test_membership test
        uses: actions/download-artifact@v8
        with:
          name: membership-report
          path: .members
          merge-multiple: true
      - name: Display structure of downloaded files
        run: ls -lha .members
 
      - name: Tox Test - Implementation
        # Note: `--skip-missing-interpreters` prevents false success if python
        # version is not installed when testing.
        run: |
          which tox
          tox --skip-missing-interpreters false -e test-begin,${{ matrix.test_env }}-impl
 
      - name: Tox Test - Caveats
        run: |
          tox --skip-missing-interpreters false -e ${{ matrix.test_env }}-caveats
 
      - name: Tox Test - Examples
        run: |
          tox --skip-missing-interpreters false -e ${{ matrix.test_env }}-examples
 
  pip-package:
    timeout-minutes: 30
    # Build and upload pip packages as artifacts so we can inspect them and
    # ensure they are correct for actual release
    runs-on: latchkey-small
 
    steps:
      - name: Checkout repository
        uses: actions/checkout@v6
      - name: Set up Python
        uses: actions/setup-python@v6
        with:
          cache: 'pip'
          python-version: "3.10.x"
      - name: Install build dependency
        run: python3 -m pip install --upgrade build
      - name: Build a binary wheel and a source tarball
        run: python3 -m build
      - name: Upload packages.
        uses: actions/upload-artifact@v7
        with:
          name: pip-packages
          path: |
            dist/qt[._]py-*.whl
            dist/qt[._]py-*.tar.gz
 

What changed

What Latchkey heals here

This workflow has steps that commonly fail on transient issues (network, registries, flaky browsers). On Latchkey managed runners they are detected, retried, and self-healed instead of failing your build:

This workflow runs 6 jobs (21 with the matrix expanded) per trigger. On Latchkey the same minutes cost up to 58% less than GitHub-hosted, with zero queue time.

Actions used in this workflow