build workflow (microsoft/maro)
The build workflow from microsoft/maro, explained and optimized by Latchkey.
F
CI health: F - at risk
Run this on Latchkey for self-healing, caching, and up to 58% lower cost.
Grade your own workflow free or run it on Latchkey →What it does
This is the build workflow from the microsoft/maro repository, a real project running GitHub Actions. It is shown here with attribution under its MIT license.
Below, Latchkey shows a faster, safer version produced by its optimization engine.
The workflow
workflow (.yml)
name: build
on:
push:
tags:
- "maro-[0-9]+\\.[0-9]+\\.*"
workflow_dispatch:
jobs:
build:
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-latest, windows-latest, macos-latest]
python-version: [3.7, 3.8, 3.9]
steps:
- uses: actions/checkout@v2
- name: setup python
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install wheel twine
- name: Install build dependencies
run: |
pip install -r maro/requirements.build.txt
- name: Compile cython files
run: |
cython ./maro/backends/backend.pyx ./maro/backends/np_backend.pyx ./maro/backends/raw_backend.pyx ./maro/backends/frame.pyx --cplus -3 -E NODES_MEMORY_LAYOUT=ONE_BLOCK -X embedsignature=True
- name: Build wheel on Windows and macOS
if: runner.os == 'Windows' || runner.os == 'macOS'
run: |
python setup.py bdist_wheel
- name: Build source package on Windows
if: runner.os == 'Windows' && matrix.python-version == '3.7'
run: |
python setup.py sdist
- name: Build manylinux wheel
if: runner.os == 'Linux' && matrix.python-version == '3.7'
uses: RalfG/python-wheels-manylinux-build@v0.3.1-manylinux2010_x86_64
env:
GITHUB_BUILD_ACTION: True
with:
python-versions: 'cp37-cp37m cp38-cp38 cp39-cp39'
build-requirements: 'numpy==1.21.6'
system-packages: 'libffi-devel'
pre-build-command: 'export LD_LIBRARY_PATH=/usr/local/lib/:$LD_LIBRARY_PATH'
pip-wheel-args: '-w ./wheelhouse' # save wheel packages to wheelhouse folder
- name: Move valid packages to dist folder for manylinux
if: runner.os == 'Linux' && matrix.python-version == '3.7'
run: |
mkdir -p dist
cp wheelhouse/pymaro-*-manylinux*.whl dist
- name: Upload linux wheel to artifact
uses: actions/upload-artifact@v2
with:
name: local-wheels
path: |
dist/*.whl
dist/*.gz
- name: Publish wheels to PyPI
env:
TWINE_USERNAME: __token__
TWINE_PASSWORD: ${{ secrets.PYPI_TOKEN }}
if: (runner.os == 'Linux' && matrix.python-version == '3.7') || (runner.os == 'Windows' || runner.os == 'macOS')
run: |
twine upload --verbose dist/*.whl
The same workflow, on Latchkey
Estimated ~20% faster on cache hits, plus fewer wasted runs and a safer supply chain. Added and changed lines are highlighted.
name: build on: push: tags: - "maro-[0-9]+\\.[0-9]+\\.*" workflow_dispatch: concurrency: group: ${{ github.workflow }}-${{ github.ref }} cancel-in-progress: true jobs: build: timeout-minutes: 30 runs-on: ${{ matrix.os }} strategy: matrix: os: [ubuntu-latest, windows-latest, macos-latest] python-version: [3.7, 3.8, 3.9] steps: - uses: actions/checkout@v2 - name: setup python uses: actions/setup-python@v2 with: cache: 'pip' python-version: ${{ matrix.python-version }} - name: Install dependencies run: | python -m pip install --upgrade pip pip install wheel twine - name: Install build dependencies run: | pip install -r maro/requirements.build.txt - name: Compile cython files run: | cython ./maro/backends/backend.pyx ./maro/backends/np_backend.pyx ./maro/backends/raw_backend.pyx ./maro/backends/frame.pyx --cplus -3 -E NODES_MEMORY_LAYOUT=ONE_BLOCK -X embedsignature=True - name: Build wheel on Windows and macOS if: runner.os == 'Windows' || runner.os == 'macOS' run: | python setup.py bdist_wheel - name: Build source package on Windows if: runner.os == 'Windows' && matrix.python-version == '3.7' run: | python setup.py sdist - name: Build manylinux wheel if: runner.os == 'Linux' && matrix.python-version == '3.7' uses: RalfG/python-wheels-manylinux-build@v0.3.1-manylinux2010_x86_64 env: GITHUB_BUILD_ACTION: True with: python-versions: 'cp37-cp37m cp38-cp38 cp39-cp39' build-requirements: 'numpy==1.21.6' system-packages: 'libffi-devel' pre-build-command: 'export LD_LIBRARY_PATH=/usr/local/lib/:$LD_LIBRARY_PATH' pip-wheel-args: '-w ./wheelhouse' # save wheel packages to wheelhouse folder - name: Move valid packages to dist folder for manylinux if: runner.os == 'Linux' && matrix.python-version == '3.7' run: | mkdir -p dist cp wheelhouse/pymaro-*-manylinux*.whl dist - name: Upload linux wheel to artifact uses: actions/upload-artifact@v2 with: name: local-wheels path: | dist/*.whl dist/*.gz - name: Publish wheels to PyPI env: TWINE_USERNAME: __token__ TWINE_PASSWORD: ${{ secrets.PYPI_TOKEN }} if: (runner.os == 'Linux' && matrix.python-version == '3.7') || (runner.os == 'Windows' || runner.os == 'macOS') run: | twine upload --verbose dist/*.whl
What changed
- Cancel superseded runs when a branch or PR gets a newer push.
- Cache dependency installs on the setup step so they are served from cache.
- Add a job timeout so a hung step cannot burn hours of runner time.
1 third-party action is referenced by a movable tag. Pin it to the commit SHA (Latchkey resolves and applies this automatically) so a repointed tag cannot change what runs.
What Latchkey heals here
This workflow has steps that commonly fail on transient issues (network, registries, flaky browsers). On Latchkey managed runners they are detected, retried, and self-healed instead of failing your build:
- Dependency installs
This workflow runs 1 job (9 with the matrix expanded) per trigger. On Latchkey the same minutes cost up to 58% less than GitHub-hosted, with zero queue time.