Skip to content
Latchkey

tests workflow (explosion/thinc)

The tests workflow from explosion/thinc, explained and optimized by Latchkey.

C

CI health: C - fair

Point runs-on at Latchkey and get caching, job timeouts, self-healing for flaky steps, and up to 58% lower cost, applied automatically.

Grade your own workflow free or run it on Latchkey →
Source: explosion/thinc.github/workflows/tests.ymlLicense MITView source

What it does

This is the tests workflow from the explosion/thinc repository, a real project running GitHub Actions. It is shown here with attribution under its MIT license.

Below, Latchkey shows a faster, safer version produced by its optimization engine.

The workflow

workflow (.yml)
name: tests

on:
  push:
    branches: [main, v8.3.x]
    paths-ignore:
      - "website/**"
      - "*.md"
  pull_request:
    branches: ["*"]
    paths-ignore:
      - "website/**"
      - "*.md"
  workflow_dispatch: # allows you to trigger manually

permissions:
  contents: read

# When this workflow is queued, automatically cancel any previous running
# or pending jobs from the same branch
concurrency:
  group: tests-${{ github.ref }}
  cancel-in-progress: true

jobs:
  validate:
    name: Validate
    runs-on: ubuntu-latest
    steps:
      - name: Check out repo
        uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6

      - name: Configure Python version
        uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6
        with:
          python-version: "3.10"

      - name: black
        run: |
          python -m pip install black -c requirements.txt
          python -m black thinc --check
      - name: isort
        run: |
          python -m pip install isort -c requirements.txt
          python -m isort thinc --check
      - name: flake8
        run: |
          python -m pip install flake8 -c requirements.txt
          python -m flake8 thinc --count --select=E901,E999,F821,F822,F823,W605 --show-source --statistics

  tests:
    name: ${{ matrix.os }} - Python ${{ matrix.python_version }}
    strategy:
      fail-fast: false
      matrix:
        os:
          - ubuntu-latest
          - ubuntu-24.04-arm
          - macos-15-intel
          - macos-latest
          - windows-latest
          - windows-11-arm
        python_version: ["3.10", "3.11", "3.12", "3.13"]
        exclude:
          - os: windows-11-arm
            python_version: "3.10"

    runs-on: ${{ matrix.os }}
    env:
      NOTEBOOK_KERNEL: "thinc-notebook-tests"

    steps:
      - name: Check out repo
        uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6

      - name: Configure Python version
        uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6
        with:
          python-version: ${{ matrix.python_version }}

      - name: Install build dependencies
        run: python -m pip install --upgrade build pip wheel

      - name: Build sdist and wheel
        run: python -m build

      - name: Delete source directory
        run: rm -rf thinc
        shell: bash

      - name: Uninstall all packages
        run: |
          python -m pip freeze
          pip freeze --exclude pywin32 > installed.txt
          pip uninstall -y -r installed.txt

      - name: Install from wheel
        run: pip install dist/*.whl
        shell: bash

      - name: Test import
        run: python -c "import thinc"

      - name: Install test requirements
        run: pip install -r requirements.txt

      - name: Install notebook test requirements
        run: python -m ipykernel install --name thinc-notebook-tests --user

      - name: List installed packages
        run: python -m pip list

      - name: Run tests without extras
        run: python -m pytest --pyargs thinc --cov=thinc --cov-report=term

      # Notes on numpy requirements hacks:
      # 1. torch does not have a direct numpy requirement but is compiled
      # against a newer version than the oldest supported numpy for windows and
      # python 3.10; this version of numpy would not work with
      # tensorflow~=2.5.0 as specified above, but there is no release for
      # python 3.10 anyway
      # 2. restrict to numpy<1.24.0 due to mxnet incompatibility
      # 3. forbid torch!=1.13.0 due to segfaults with numpy<1.24.0
      # Note: some of these pip install commands are known to fail for some platforms.
      # To continue despite errors as in azure pipelines, remove -e from the default
      # bash flags.
      #- name: Install extras for testing
      #  run: |
      #    pip install "protobuf~=3.20.0" "tensorflow~=2.5.0"
      #    pip install "mxnet; sys_platform != 'win32' and python_version < '3.12'"
      #    pip install "torch!=1.13.0; sys_platform!='darwin'" --extra-index-url https://download.pytorch.org/whl/cpu
      #    # there is a bug related to MPS devices in github macos runners that
      #    # will be fixed in torch v2.1.1
      #    # https://github.com/pytorch/pytorch/pull/111576
      #    pip install "torch>=2.1.1; sys_platform=='darwin'" --extra-index-url https://download.pytorch.org/whl/cpu
      #    pip install "numpy~=1.23.0; python_version=='3.10' and sys_platform=='win32'"
      #    pip install "numpy<1.24.0"
      #    pip install -r requirements.txt
      #    pip uninstall -y mypy
      #  shell: bash --noprofile --norc -o pipefail {0}

      - name: Run tests with extras
        run: python -m pytest --pyargs thinc --cov=thinc --cov-report=term -p thinc.tests.enable_tensorflow -p thinc.tests.enable_mxnet

      - name: Run tests for thinc-apple-ops
        run: |
          pip uninstall -y tensorflow
          pip install "thinc-apple-ops>=1.0.0,<2.0.0"
          python -m pytest --pyargs thinc_apple_ops
        if: runner.os == 'macOS' && matrix.python_version == '3.10'

      - name: Run tests with thinc-apple-ops
        run: python -m pytest --pyargs thinc
        if: runner.os == 'macOS' && matrix.python_version == '3.10'

The same workflow, on Latchkey

Estimated ~20% faster on cache hits, plus fewer wasted runs and a safer supply chain. Added and changed lines are highlighted.

name: tests
 
on:
  push:
    branches: [main, v8.3.x]
    paths-ignore:
      - "website/**"
      - "*.md"
  pull_request:
    branches: ["*"]
    paths-ignore:
      - "website/**"
      - "*.md"
  workflow_dispatch: # allows you to trigger manually
 
permissions:
  contents: read
 
# When this workflow is queued, automatically cancel any previous running
# or pending jobs from the same branch
concurrency:
  group: tests-${{ github.ref }}
  cancel-in-progress: true
 
jobs:
  validate:
    timeout-minutes: 30
    name: Validate
    runs-on: latchkey-small
    steps:
      - name: Check out repo
        uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
 
      - name: Configure Python version
        uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6
        with:
          cache: 'pip'
          python-version: "3.10"
 
      - name: black
        run: |
          python -m pip install black -c requirements.txt
          python -m black thinc --check
      - name: isort
        run: |
          python -m pip install isort -c requirements.txt
          python -m isort thinc --check
      - name: flake8
        run: |
          python -m pip install flake8 -c requirements.txt
          python -m flake8 thinc --count --select=E901,E999,F821,F822,F823,W605 --show-source --statistics
 
  tests:
    timeout-minutes: 30
    name: ${{ matrix.os }} - Python ${{ matrix.python_version }}
    strategy:
      fail-fast: false
      matrix:
        os:
          - ubuntu-latest
          - ubuntu-24.04-arm
          - macos-15-intel
          - macos-latest
          - windows-latest
          - windows-11-arm
        python_version: ["3.10", "3.11", "3.12", "3.13"]
        exclude:
          - os: windows-11-arm
            python_version: "3.10"
 
    runs-on: ${{ matrix.os }}
    env:
      NOTEBOOK_KERNEL: "thinc-notebook-tests"
 
    steps:
      - name: Check out repo
        uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
 
      - name: Configure Python version
        uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6
        with:
          cache: 'pip'
          python-version: ${{ matrix.python_version }}
 
      - name: Install build dependencies
        run: python -m pip install --upgrade build pip wheel
 
      - name: Build sdist and wheel
        run: python -m build
 
      - name: Delete source directory
        run: rm -rf thinc
        shell: bash
 
      - name: Uninstall all packages
        run: |
          python -m pip freeze
          pip freeze --exclude pywin32 > installed.txt
          pip uninstall -y -r installed.txt
 
      - name: Install from wheel
        run: pip install dist/*.whl
        shell: bash
 
      - name: Test import
        run: python -c "import thinc"
 
      - name: Install test requirements
        run: pip install -r requirements.txt
 
      - name: Install notebook test requirements
        run: python -m ipykernel install --name thinc-notebook-tests --user
 
      - name: List installed packages
        run: python -m pip list
 
      - name: Run tests without extras
        run: python -m pytest --pyargs thinc --cov=thinc --cov-report=term
 
      # Notes on numpy requirements hacks:
      # 1. torch does not have a direct numpy requirement but is compiled
      # against a newer version than the oldest supported numpy for windows and
      # python 3.10; this version of numpy would not work with
      # tensorflow~=2.5.0 as specified above, but there is no release for
      # python 3.10 anyway
      # 2. restrict to numpy<1.24.0 due to mxnet incompatibility
      # 3. forbid torch!=1.13.0 due to segfaults with numpy<1.24.0
      # Note: some of these pip install commands are known to fail for some platforms.
      # To continue despite errors as in azure pipelines, remove -e from the default
      # bash flags.
      #- name: Install extras for testing
      #  run: |
      #    pip install "protobuf~=3.20.0" "tensorflow~=2.5.0"
      #    pip install "mxnet; sys_platform != 'win32' and python_version < '3.12'"
      #    pip install "torch!=1.13.0; sys_platform!='darwin'" --extra-index-url https://download.pytorch.org/whl/cpu
      #    # there is a bug related to MPS devices in github macos runners that
      #    # will be fixed in torch v2.1.1
      #    # https://github.com/pytorch/pytorch/pull/111576
      #    pip install "torch>=2.1.1; sys_platform=='darwin'" --extra-index-url https://download.pytorch.org/whl/cpu
      #    pip install "numpy~=1.23.0; python_version=='3.10' and sys_platform=='win32'"
      #    pip install "numpy<1.24.0"
      #    pip install -r requirements.txt
      #    pip uninstall -y mypy
      #  shell: bash --noprofile --norc -o pipefail {0}
 
      - name: Run tests with extras
        run: python -m pytest --pyargs thinc --cov=thinc --cov-report=term -p thinc.tests.enable_tensorflow -p thinc.tests.enable_mxnet
 
      - name: Run tests for thinc-apple-ops
        run: |
          pip uninstall -y tensorflow
          pip install "thinc-apple-ops>=1.0.0,<2.0.0"
          python -m pytest --pyargs thinc_apple_ops
        if: runner.os == 'macOS' && matrix.python_version == '3.10'
 
      - name: Run tests with thinc-apple-ops
        run: python -m pytest --pyargs thinc
        if: runner.os == 'macOS' && matrix.python_version == '3.10'
 

What changed

What Latchkey heals here

This workflow has steps that commonly fail on transient issues (network, registries, flaky browsers). On Latchkey managed runners they are detected, retried, and self-healed instead of failing your build:

This workflow runs 2 jobs (25 with the matrix expanded) per trigger. On Latchkey the same minutes cost up to 58% less than GitHub-hosted, with zero queue time.

Actions used in this workflow