tests workflow (explosion/thinc)
The tests workflow from explosion/thinc, explained and optimized by Latchkey.
CI health: C - fair
Point runs-on at Latchkey and get caching, job timeouts, self-healing for flaky steps, and up to 58% lower cost, applied automatically.
What it does
This is the tests workflow from the explosion/thinc repository, a real project running GitHub Actions. It is shown here with attribution under its MIT license.
Below, Latchkey shows a faster, safer version produced by its optimization engine.
The workflow
name: tests
on:
push:
branches: [main, v8.3.x]
paths-ignore:
- "website/**"
- "*.md"
pull_request:
branches: ["*"]
paths-ignore:
- "website/**"
- "*.md"
workflow_dispatch: # allows you to trigger manually
permissions:
contents: read
# When this workflow is queued, automatically cancel any previous running
# or pending jobs from the same branch
concurrency:
group: tests-${{ github.ref }}
cancel-in-progress: true
jobs:
validate:
name: Validate
runs-on: ubuntu-latest
steps:
- name: Check out repo
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- name: Configure Python version
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6
with:
python-version: "3.10"
- name: black
run: |
python -m pip install black -c requirements.txt
python -m black thinc --check
- name: isort
run: |
python -m pip install isort -c requirements.txt
python -m isort thinc --check
- name: flake8
run: |
python -m pip install flake8 -c requirements.txt
python -m flake8 thinc --count --select=E901,E999,F821,F822,F823,W605 --show-source --statistics
tests:
name: ${{ matrix.os }} - Python ${{ matrix.python_version }}
strategy:
fail-fast: false
matrix:
os:
- ubuntu-latest
- ubuntu-24.04-arm
- macos-15-intel
- macos-latest
- windows-latest
- windows-11-arm
python_version: ["3.10", "3.11", "3.12", "3.13"]
exclude:
- os: windows-11-arm
python_version: "3.10"
runs-on: ${{ matrix.os }}
env:
NOTEBOOK_KERNEL: "thinc-notebook-tests"
steps:
- name: Check out repo
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- name: Configure Python version
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6
with:
python-version: ${{ matrix.python_version }}
- name: Install build dependencies
run: python -m pip install --upgrade build pip wheel
- name: Build sdist and wheel
run: python -m build
- name: Delete source directory
run: rm -rf thinc
shell: bash
- name: Uninstall all packages
run: |
python -m pip freeze
pip freeze --exclude pywin32 > installed.txt
pip uninstall -y -r installed.txt
- name: Install from wheel
run: pip install dist/*.whl
shell: bash
- name: Test import
run: python -c "import thinc"
- name: Install test requirements
run: pip install -r requirements.txt
- name: Install notebook test requirements
run: python -m ipykernel install --name thinc-notebook-tests --user
- name: List installed packages
run: python -m pip list
- name: Run tests without extras
run: python -m pytest --pyargs thinc --cov=thinc --cov-report=term
# Notes on numpy requirements hacks:
# 1. torch does not have a direct numpy requirement but is compiled
# against a newer version than the oldest supported numpy for windows and
# python 3.10; this version of numpy would not work with
# tensorflow~=2.5.0 as specified above, but there is no release for
# python 3.10 anyway
# 2. restrict to numpy<1.24.0 due to mxnet incompatibility
# 3. forbid torch!=1.13.0 due to segfaults with numpy<1.24.0
# Note: some of these pip install commands are known to fail for some platforms.
# To continue despite errors as in azure pipelines, remove -e from the default
# bash flags.
#- name: Install extras for testing
# run: |
# pip install "protobuf~=3.20.0" "tensorflow~=2.5.0"
# pip install "mxnet; sys_platform != 'win32' and python_version < '3.12'"
# pip install "torch!=1.13.0; sys_platform!='darwin'" --extra-index-url https://download.pytorch.org/whl/cpu
# # there is a bug related to MPS devices in github macos runners that
# # will be fixed in torch v2.1.1
# # https://github.com/pytorch/pytorch/pull/111576
# pip install "torch>=2.1.1; sys_platform=='darwin'" --extra-index-url https://download.pytorch.org/whl/cpu
# pip install "numpy~=1.23.0; python_version=='3.10' and sys_platform=='win32'"
# pip install "numpy<1.24.0"
# pip install -r requirements.txt
# pip uninstall -y mypy
# shell: bash --noprofile --norc -o pipefail {0}
- name: Run tests with extras
run: python -m pytest --pyargs thinc --cov=thinc --cov-report=term -p thinc.tests.enable_tensorflow -p thinc.tests.enable_mxnet
- name: Run tests for thinc-apple-ops
run: |
pip uninstall -y tensorflow
pip install "thinc-apple-ops>=1.0.0,<2.0.0"
python -m pytest --pyargs thinc_apple_ops
if: runner.os == 'macOS' && matrix.python_version == '3.10'
- name: Run tests with thinc-apple-ops
run: python -m pytest --pyargs thinc
if: runner.os == 'macOS' && matrix.python_version == '3.10'
The same workflow, on Latchkey
Estimated ~20% faster on cache hits, plus fewer wasted runs and a safer supply chain. Added and changed lines are highlighted.
name: tests on: push: branches: [main, v8.3.x] paths-ignore: - "website/**" - "*.md" pull_request: branches: ["*"] paths-ignore: - "website/**" - "*.md" workflow_dispatch: # allows you to trigger manually permissions: contents: read # When this workflow is queued, automatically cancel any previous running # or pending jobs from the same branch concurrency: group: tests-${{ github.ref }} cancel-in-progress: true jobs: validate: timeout-minutes: 30 name: Validate runs-on: latchkey-small steps: - name: Check out repo uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6 - name: Configure Python version uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6 with: cache: 'pip' python-version: "3.10" - name: black run: | python -m pip install black -c requirements.txt python -m black thinc --check - name: isort run: | python -m pip install isort -c requirements.txt python -m isort thinc --check - name: flake8 run: | python -m pip install flake8 -c requirements.txt python -m flake8 thinc --count --select=E901,E999,F821,F822,F823,W605 --show-source --statistics tests: timeout-minutes: 30 name: ${{ matrix.os }} - Python ${{ matrix.python_version }} strategy: fail-fast: false matrix: os: - ubuntu-latest - ubuntu-24.04-arm - macos-15-intel - macos-latest - windows-latest - windows-11-arm python_version: ["3.10", "3.11", "3.12", "3.13"] exclude: - os: windows-11-arm python_version: "3.10" runs-on: ${{ matrix.os }} env: NOTEBOOK_KERNEL: "thinc-notebook-tests" steps: - name: Check out repo uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6 - name: Configure Python version uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6 with: cache: 'pip' python-version: ${{ matrix.python_version }} - name: Install build dependencies run: python -m pip install --upgrade build pip wheel - name: Build sdist and wheel run: python -m build - name: Delete source directory run: rm -rf thinc shell: bash - name: Uninstall all packages run: | python -m pip freeze pip freeze --exclude pywin32 > installed.txt pip uninstall -y -r installed.txt - name: Install from wheel run: pip install dist/*.whl shell: bash - name: Test import run: python -c "import thinc" - name: Install test requirements run: pip install -r requirements.txt - name: Install notebook test requirements run: python -m ipykernel install --name thinc-notebook-tests --user - name: List installed packages run: python -m pip list - name: Run tests without extras run: python -m pytest --pyargs thinc --cov=thinc --cov-report=term # Notes on numpy requirements hacks: # 1. torch does not have a direct numpy requirement but is compiled # against a newer version than the oldest supported numpy for windows and # python 3.10; this version of numpy would not work with # tensorflow~=2.5.0 as specified above, but there is no release for # python 3.10 anyway # 2. restrict to numpy<1.24.0 due to mxnet incompatibility # 3. forbid torch!=1.13.0 due to segfaults with numpy<1.24.0 # Note: some of these pip install commands are known to fail for some platforms. # To continue despite errors as in azure pipelines, remove -e from the default # bash flags. #- name: Install extras for testing # run: | # pip install "protobuf~=3.20.0" "tensorflow~=2.5.0" # pip install "mxnet; sys_platform != 'win32' and python_version < '3.12'" # pip install "torch!=1.13.0; sys_platform!='darwin'" --extra-index-url https://download.pytorch.org/whl/cpu # # there is a bug related to MPS devices in github macos runners that # # will be fixed in torch v2.1.1 # # https://github.com/pytorch/pytorch/pull/111576 # pip install "torch>=2.1.1; sys_platform=='darwin'" --extra-index-url https://download.pytorch.org/whl/cpu # pip install "numpy~=1.23.0; python_version=='3.10' and sys_platform=='win32'" # pip install "numpy<1.24.0" # pip install -r requirements.txt # pip uninstall -y mypy # shell: bash --noprofile --norc -o pipefail {0} - name: Run tests with extras run: python -m pytest --pyargs thinc --cov=thinc --cov-report=term -p thinc.tests.enable_tensorflow -p thinc.tests.enable_mxnet - name: Run tests for thinc-apple-ops run: | pip uninstall -y tensorflow pip install "thinc-apple-ops>=1.0.0,<2.0.0" python -m pytest --pyargs thinc_apple_ops if: runner.os == 'macOS' && matrix.python_version == '3.10' - name: Run tests with thinc-apple-ops run: python -m pytest --pyargs thinc if: runner.os == 'macOS' && matrix.python_version == '3.10'
What changed
- Run on Latchkey managed runners with one line (
runs-on), which apply the fixes below automatically and self-heal transient failures. This example useslatchkey-small; pick the runner size that fits the job. - Cache dependency installs on the setup step so they are served from cache.
- Add a job timeout so a hung step cannot burn hours of runner time.
What Latchkey heals here
This workflow has steps that commonly fail on transient issues (network, registries, flaky browsers). On Latchkey managed runners they are detected, retried, and self-healed instead of failing your build:
- Dependency installs
This workflow runs 2 jobs (25 with the matrix expanded) per trigger. On Latchkey the same minutes cost up to 58% less than GitHub-hosted, with zero queue time.