Build workflow (DigitalSlideArchive/HistomicsTK)
The Build workflow from DigitalSlideArchive/HistomicsTK, explained and optimized by Latchkey.
CI health: C - fair
Point runs-on at Latchkey and get caching, job timeouts, SHA-pinned actions, self-healing for flaky steps, and up to 58% lower cost, applied automatically.
What it does
This is the Build workflow from the DigitalSlideArchive/HistomicsTK repository, a real project running GitHub Actions. It is shown here with attribution under its Apache-2.0 license.
Below, Latchkey shows a faster, safer version produced by its optimization engine.
The workflow
name: Build
on:
push:
# pull_request:
# branches:
# - master
release:
types:
- created
# schedule:
# - cron: "0 7 * * 1"
concurrency:
group: ${{ github.ref }}
cancel-in-progress: ${{ github.ref != 'refs/heads/master' }}
jobs:
test:
runs-on: ubuntu-latest
# Runs can be slow, so don't use resources unless we mean to publish
if: github.repository == 'DigitalSlideArchive/HistomicsTK' && ( startsWith(github.ref, 'refs/tags/v') || github.ref == 'refs/heads/master' )
strategy:
matrix:
python: ["3.10", "3.11", "3.12", "3.13", "3.14"]
steps:
- uses: actions/checkout@v7
- uses: actions/setup-python@v6
with:
python-version: ${{ matrix.python }}
- name: Install tox
run: |
pip install --upgrade pip
pip install tox tox-gh-actions
- name: Run tox
env:
PYTEST_ADDOPTS: "--durations=0 -vv"
run: |
tox
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: actions/setup-python@v6
with:
python-version: "3.11"
- uses: nikeee/setup-pandoc@v1
- name: Install tox
run: |
pip install --upgrade pip
pip install tox
- name: Run tox for lint and docs
run: |
tox -e lint,docs
- uses: actions/upload-artifact@v7
with:
name: docs
path: ./docs/_build
build_wheels:
name: Wheels for ${{ matrix.python }} / ${{ matrix.buildplat[0] }} / ${{ matrix.buildplat[1] }}
runs-on: ${{ matrix.buildplat[0] }}
# Don't build external PRs.
if: github.repository == 'DigitalSlideArchive/HistomicsTK'
strategy:
matrix:
# See matplotlib for some of the rationale of this
buildplat:
- [ubuntu-latest, "x86_64 aarch64"]
- [macos-latest, "x86_64 arm64"]
- [windows-latest, auto64]
python: ["cp310", "cp311", "cp312", "cp313", "cp314"]
fail-fast: false
steps:
- uses: actions/checkout@v7
with:
fetch-depth: 0
fetch-tags: true
- run: git status && git tag --list && pwd
- uses: actions/setup-python@v6
with:
python-version: "3.x"
- run: python -m pip install setuptools_scm
- run: python -c "from setuptools_scm import get_version; print(get_version()); open('histomicstk/_version.py', 'w').write(f'__version__ = {str(get_version()).split('+')[0]!r}\n')"
- run: cat histomicstk/_version.py
- shell: bash
run: |
sed -i.bak -E \
-e '/^dynamic = \["version"\]/d' \
-e '/^\[project\]/a\
version = "'"$(sed -nE "s/__version__ = ['\"](.*)['\"]/\\1/p" histomicstk/_version.py)"'"' \
pyproject.toml
- run: cat pyproject.toml
- name: Set up QEMU
if: runner.os == 'Linux'
uses: docker/setup-qemu-action@v4
- name: Build wheels
uses: pypa/cibuildwheel@v4.1.0
env:
CIBW_BUILD: ${{ matrix.python }}-*
CIBW_TEST_SKIP: "*-macosx_arm64"
CIBW_ARCHS: ${{ matrix.buildplat[1] }}
MACOSX_DEPLOYMENT_TARGET: "11.0"
- uses: actions/upload-artifact@v7
with:
name: dist-${{ matrix.python }}-${{ startsWith(matrix.buildplat[1], 'macosx') && 'macosx' || matrix.buildplat[1] }}
path: ./wheelhouse/*.whl
make_sdist:
name: Make SDist
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with:
fetch-depth: 0
fetch-tags: true
- run: git status && git tag --list && pwd
- uses: actions/setup-python@v6
with:
python-version: "3.x"
- run: python -m pip install setuptools_scm
- run: python -c "from setuptools_scm import get_version; print(get_version()); open('histomicstk/_version.py', 'w').write(f'__version__ = {str(get_version()).split('+')[0]!r}\n')"
- run: cat histomicstk/_version.py
- shell: bash
run: |
sed -i.bak -E \
-e '/^dynamic = \["version"\]/d' \
-e '/^\[project\]/a\
version = "'"$(sed -nE "s/__version__ = ['\"](.*)['\"]/\\1/p" histomicstk/_version.py)"'"' \
pyproject.toml
- run: cat pyproject.toml
- name: Build SDist
run: pipx run build --sdist
- uses: actions/upload-artifact@v7
with:
name: dist-sdist
path: dist/*.tar.gz
# Still on Circle-CI
# - docker
# - publish-docker
# - docs-deploy
test-release:
runs-on: ubuntu-latest
needs:
- lint
- build_wheels
- make_sdist
steps:
- uses: actions/download-artifact@v8
with:
path: dist
pattern: dist-*
merge-multiple: true
- name: List dist directory
run: ls dist
release:
runs-on: ubuntu-latest
needs:
- test
- lint
- build_wheels
- make_sdist
if: github.repository == 'DigitalSlideArchive/HistomicsTK' && ( startsWith(github.ref, 'refs/tags/v') || github.ref == 'refs/heads/master' )
environment:
name: pypi
url: https://pypi.org/p/histomicstk
permissions:
id-token: write
steps:
- uses: actions/download-artifact@v8
with:
path: dist
pattern: dist-*
merge-multiple: true
- name: List dist directory
run: ls dist
- uses: pypa/gh-action-pypi-publish@release/v1
The same workflow, on Latchkey
Estimated ~20% faster on cache hits, plus fewer wasted runs and a safer supply chain. Added and changed lines are highlighted.
name: Build on: push: # pull_request: # branches: # - master release: types: - created # schedule: # - cron: "0 7 * * 1" concurrency: group: ${{ github.ref }} cancel-in-progress: ${{ github.ref != 'refs/heads/master' }} jobs: test: timeout-minutes: 30 runs-on: latchkey-small # Runs can be slow, so don't use resources unless we mean to publish if: github.repository == 'DigitalSlideArchive/HistomicsTK' && ( startsWith(github.ref, 'refs/tags/v') || github.ref == 'refs/heads/master' ) strategy: matrix: python: ["3.10", "3.11", "3.12", "3.13", "3.14"] steps: - uses: actions/checkout@v7 - uses: actions/setup-python@v6 with: cache: 'pip' python-version: ${{ matrix.python }} - name: Install tox run: | pip install --upgrade pip pip install tox tox-gh-actions - name: Run tox env: PYTEST_ADDOPTS: "--durations=0 -vv" run: | tox lint: timeout-minutes: 30 runs-on: latchkey-small steps: - uses: actions/checkout@v7 - uses: actions/setup-python@v6 with: cache: 'pip' python-version: "3.11" - uses: nikeee/setup-pandoc@v1 - name: Install tox run: | pip install --upgrade pip pip install tox - name: Run tox for lint and docs run: | tox -e lint,docs - uses: actions/upload-artifact@v7 with: name: docs path: ./docs/_build build_wheels: timeout-minutes: 30 name: Wheels for ${{ matrix.python }} / ${{ matrix.buildplat[0] }} / ${{ matrix.buildplat[1] }} runs-on: ${{ matrix.buildplat[0] }} # Don't build external PRs. if: github.repository == 'DigitalSlideArchive/HistomicsTK' strategy: matrix: # See matplotlib for some of the rationale of this buildplat: - [ubuntu-latest, "x86_64 aarch64"] - [macos-latest, "x86_64 arm64"] - [windows-latest, auto64] python: ["cp310", "cp311", "cp312", "cp313", "cp314"] fail-fast: false steps: - uses: actions/checkout@v7 with: fetch-depth: 0 fetch-tags: true - run: git status && git tag --list && pwd - uses: actions/setup-python@v6 with: cache: 'pip' python-version: "3.x" - run: python -m pip install setuptools_scm - run: python -c "from setuptools_scm import get_version; print(get_version()); open('histomicstk/_version.py', 'w').write(f'__version__ = {str(get_version()).split('+')[0]!r}\n')" - run: cat histomicstk/_version.py - shell: bash run: | sed -i.bak -E \ -e '/^dynamic = \["version"\]/d' \ -e '/^\[project\]/a\ version = "'"$(sed -nE "s/__version__ = ['\"](.*)['\"]/\\1/p" histomicstk/_version.py)"'"' \ pyproject.toml - run: cat pyproject.toml - name: Set up QEMU if: runner.os == 'Linux' uses: docker/setup-qemu-action@v4 - name: Build wheels uses: pypa/cibuildwheel@v4.1.0 env: CIBW_BUILD: ${{ matrix.python }}-* CIBW_TEST_SKIP: "*-macosx_arm64" CIBW_ARCHS: ${{ matrix.buildplat[1] }} MACOSX_DEPLOYMENT_TARGET: "11.0" - uses: actions/upload-artifact@v7 with: name: dist-${{ matrix.python }}-${{ startsWith(matrix.buildplat[1], 'macosx') && 'macosx' || matrix.buildplat[1] }} path: ./wheelhouse/*.whl make_sdist: timeout-minutes: 30 name: Make SDist runs-on: latchkey-small steps: - uses: actions/checkout@v7 with: fetch-depth: 0 fetch-tags: true - run: git status && git tag --list && pwd - uses: actions/setup-python@v6 with: cache: 'pip' python-version: "3.x" - run: python -m pip install setuptools_scm - run: python -c "from setuptools_scm import get_version; print(get_version()); open('histomicstk/_version.py', 'w').write(f'__version__ = {str(get_version()).split('+')[0]!r}\n')" - run: cat histomicstk/_version.py - shell: bash run: | sed -i.bak -E \ -e '/^dynamic = \["version"\]/d' \ -e '/^\[project\]/a\ version = "'"$(sed -nE "s/__version__ = ['\"](.*)['\"]/\\1/p" histomicstk/_version.py)"'"' \ pyproject.toml - run: cat pyproject.toml - name: Build SDist run: pipx run build --sdist - uses: actions/upload-artifact@v7 with: name: dist-sdist path: dist/*.tar.gz # Still on Circle-CI # - docker # - publish-docker # - docs-deploy test-release: timeout-minutes: 30 runs-on: latchkey-small needs: - lint - build_wheels - make_sdist steps: - uses: actions/download-artifact@v8 with: path: dist pattern: dist-* merge-multiple: true - name: List dist directory run: ls dist release: timeout-minutes: 30 runs-on: latchkey-small needs: - test - lint - build_wheels - make_sdist if: github.repository == 'DigitalSlideArchive/HistomicsTK' && ( startsWith(github.ref, 'refs/tags/v') || github.ref == 'refs/heads/master' ) environment: name: pypi url: https://pypi.org/p/histomicstk permissions: id-token: write steps: - uses: actions/download-artifact@v8 with: path: dist pattern: dist-* merge-multiple: true - name: List dist directory run: ls dist - uses: pypa/gh-action-pypi-publish@release/v1
What changed
- Run on Latchkey managed runners with one line (
runs-on), which apply the fixes below automatically and self-heal transient failures. This example useslatchkey-small; pick the runner size that fits the job. - Cache dependency installs on the setup step so they are served from cache.
- Add a job timeout so a hung step cannot burn hours of runner time.
4 third-party actions are referenced by a movable tag. Pin them to the commit SHA (Latchkey resolves and applies this automatically) so a repointed tag cannot change what runs.
What Latchkey heals here
This workflow has steps that commonly fail on transient issues (network, registries, flaky browsers). On Latchkey managed runners they are detected, retried, and self-healed instead of failing your build:
- Dependency installs
This workflow runs 6 jobs (24 with the matrix expanded) per trigger. On Latchkey the same minutes cost up to 58% less than GitHub-hosted, with zero queue time.