Build wheels workflow (hatchet/hatchet)
The Build wheels workflow from hatchet/hatchet, explained and optimized by Latchkey.
CI health: C - fair
Point runs-on at Latchkey and get run de-duplication, job timeouts, SHA-pinned actions, self-healing for flaky steps, and up to 58% lower cost, applied automatically.
What it does
This is the Build wheels workflow from the hatchet/hatchet repository, a real project running GitHub Actions. It is shown here with attribution under its MIT license.
Below, Latchkey shows a faster, safer version produced by its optimization engine.
The workflow
# Github Actions Workflow for building wheels
# The step for creating a github release is borrowed from cython
# https://github.com/cython/cython/blob/master/.github/workflows/wheels.yml
# The general structure of the workflow is also highly inspired by pandas'
# wheel builder workflow
# https://github.com/pandas-dev/pandas/blob/main/.github/workflows/wheels.yml
name: Build wheels
# Run on pull request to the develop or releases branch, but don't
# upload wheels
on:
pull_request:
types: [labeled, opened, synchronize, reopened]
branches: [develop, releases/**]
workflow_dispatch:
push:
tags:
- 'v**'
jobs:
build_wheels:
if: >-
github.event_name == 'workflow_dispatch' ||
(github.event_name == 'pull_request' &&
contains(github.event.pull_request.labels.*.name, 'type: release')) ||
(github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v'))
name: Build wheel for ${{ matrix.python[0] }}-${{ matrix.buildplat[1] }}
strategy:
fail-fast: false
matrix:
# GitHub Actions doesn't support pairing matrix values together, let's improvise
# https://github.com/github/feedback/discussions/7835#discussioncomment-1769026
# The first value of the pair is the host image
# You can find a list of host images here https://github.com/actions/runner-images
# The second is the cibuildwheel platform
# This is the second part of a CIBW_BUILD tag, the part after the cp3xx
# (e.g. see here https://cibuildwheel.readthedocs.io/en/stable/options/#build-skip)
buildplat:
- [ubuntu-22.04, manylinux_x86_64]
- [macos-12, macosx_x86_64]
# Note: M1 images on Github Actions start from macOS 14
- [macos-14, macosx_arm64]
- [windows-2022, win_amd64]
python: [["cp39", "3.9"], ["cp310", "3.10"], ["cp311", "3.11"], ["cp312", "3.12"]]
runs-on: ${{ matrix.buildplat[0] }}
steps:
- uses: actions/checkout@v4
- name: Build wheels
uses: pypa/cibuildwheel@v2.16.5
env:
# Disable build isolation since we need the installed Cython from the host
CIBW_ENVIRONMENT: PIP_NO_BUILD_ISOLATION=False
CIBW_SKIP: pp* *musllinux*
CIBW_BEFORE_BUILD: pip install cython
CIBW_BUILD: ${{ matrix.python[0] }}-${{ matrix.buildplat[1] }}
CIBW_BEFORE_TEST: pip install pytest
# TODO: re-enable tests
#CIBW_TEST_COMMAND: pytest {project}/hatchet/tests
- uses: actions/upload-artifact@v4
with:
name: ${{ matrix.python[0] }}-${{ matrix.buildplat[1] }}
path: ./wheelhouse/*.whl
- name: Release
uses: softprops/action-gh-release@v1
if: startsWith(github.ref, 'refs/tags/v')
with:
files: wheelhouse/*.whl
draft: true
prerelease: >-
${{ contains(github.ref_name, 'a') || contains(github.ref_name, 'b')
|| contains(github.ref_name, 'rc') || contains(github.ref_name, 'dev') }}
build_sdist:
if: >-
github.event_name == 'workflow_dispatch' ||
(github.event_name == 'pull_request' &&
contains(github.event.pull_request.labels.*.name, 'type: release')) ||
(github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v'))
name: Build sdist
runs-on: ubuntu-20.04
steps:
- uses: actions/checkout@v4
- name: Get PyPA build
run: python -m pip install build cython
- name: Build sdist
# Disable isolation so we can have the cython from the host
run: python -m build -s --no-isolation
- uses: actions/upload-artifact@v4
with:
name: sdist
path: dist/*.tar.gz
- name: Release
uses: softprops/action-gh-release@v1
if: startsWith(github.ref, 'refs/tags/v')
with:
files: |
dist/*.tar.gz
dist/*-none-any.whl
draft: true
prerelease: >-
${{ contains(github.ref_name, 'a') || contains(github.ref_name, 'b')
|| contains(github.ref_name, 'rc') || contains(github.ref_name, 'dev') }}
The same workflow, on Latchkey
Removes redundant runs and caps runaway jobs. Added and changed lines are highlighted.
# Github Actions Workflow for building wheels # The step for creating a github release is borrowed from cython # https://github.com/cython/cython/blob/master/.github/workflows/wheels.yml # The general structure of the workflow is also highly inspired by pandas' # wheel builder workflow # https://github.com/pandas-dev/pandas/blob/main/.github/workflows/wheels.yml name: Build wheels # Run on pull request to the develop or releases branch, but don't # upload wheels on: pull_request: types: [labeled, opened, synchronize, reopened] branches: [develop, releases/**] workflow_dispatch: push: tags: - 'v**' concurrency: group: ${{ github.workflow }}-${{ github.ref }} cancel-in-progress: true jobs: build_wheels: timeout-minutes: 30 if: >- github.event_name == 'workflow_dispatch' || (github.event_name == 'pull_request' && contains(github.event.pull_request.labels.*.name, 'type: release')) || (github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')) name: Build wheel for ${{ matrix.python[0] }}-${{ matrix.buildplat[1] }} strategy: fail-fast: false matrix: # GitHub Actions doesn't support pairing matrix values together, let's improvise # https://github.com/github/feedback/discussions/7835#discussioncomment-1769026 # The first value of the pair is the host image # You can find a list of host images here https://github.com/actions/runner-images # The second is the cibuildwheel platform # This is the second part of a CIBW_BUILD tag, the part after the cp3xx # (e.g. see here https://cibuildwheel.readthedocs.io/en/stable/options/#build-skip) buildplat: - [ubuntu-22.04, manylinux_x86_64] - [macos-12, macosx_x86_64] # Note: M1 images on Github Actions start from macOS 14 - [macos-14, macosx_arm64] - [windows-2022, win_amd64] python: [["cp39", "3.9"], ["cp310", "3.10"], ["cp311", "3.11"], ["cp312", "3.12"]] runs-on: ${{ matrix.buildplat[0] }} steps: - uses: actions/checkout@v4 - name: Build wheels uses: pypa/cibuildwheel@v2.16.5 env: # Disable build isolation since we need the installed Cython from the host CIBW_ENVIRONMENT: PIP_NO_BUILD_ISOLATION=False CIBW_SKIP: pp* *musllinux* CIBW_BEFORE_BUILD: pip install cython CIBW_BUILD: ${{ matrix.python[0] }}-${{ matrix.buildplat[1] }} CIBW_BEFORE_TEST: pip install pytest # TODO: re-enable tests #CIBW_TEST_COMMAND: pytest {project}/hatchet/tests - uses: actions/upload-artifact@v4 with: name: ${{ matrix.python[0] }}-${{ matrix.buildplat[1] }} path: ./wheelhouse/*.whl - name: Release uses: softprops/action-gh-release@v1 if: startsWith(github.ref, 'refs/tags/v') with: files: wheelhouse/*.whl draft: true prerelease: >- ${{ contains(github.ref_name, 'a') || contains(github.ref_name, 'b') || contains(github.ref_name, 'rc') || contains(github.ref_name, 'dev') }} build_sdist: timeout-minutes: 30 if: >- github.event_name == 'workflow_dispatch' || (github.event_name == 'pull_request' && contains(github.event.pull_request.labels.*.name, 'type: release')) || (github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')) name: Build sdist runs-on: latchkey-small steps: - uses: actions/checkout@v4 - name: Get PyPA build run: python -m pip install build cython - name: Build sdist # Disable isolation so we can have the cython from the host run: python -m build -s --no-isolation - uses: actions/upload-artifact@v4 with: name: sdist path: dist/*.tar.gz - name: Release uses: softprops/action-gh-release@v1 if: startsWith(github.ref, 'refs/tags/v') with: files: | dist/*.tar.gz dist/*-none-any.whl draft: true prerelease: >- ${{ contains(github.ref_name, 'a') || contains(github.ref_name, 'b') || contains(github.ref_name, 'rc') || contains(github.ref_name, 'dev') }}
What changed
- Run on Latchkey managed runners with one line (
runs-on), which apply the fixes below automatically and self-heal transient failures. This example useslatchkey-small; pick the runner size that fits the job. - Cancel superseded runs when a branch or PR gets a newer push.
- Add a job timeout so a hung step cannot burn hours of runner time.
2 third-party actions are referenced by a movable tag. Pin them to the commit SHA (Latchkey resolves and applies this automatically) so a repointed tag cannot change what runs.
What Latchkey heals here
This workflow has steps that commonly fail on transient issues (network, registries, flaky browsers). On Latchkey managed runners they are detected, retried, and self-healed instead of failing your build:
- Dependency installs
This workflow runs 2 jobs (17 with the matrix expanded) per trigger. On Latchkey the same minutes cost up to 58% less than GitHub-hosted, with zero queue time.