Deploy workflow (scikit-hep/awkward)

The Deploy workflow from scikit-hep/awkward, explained and optimized by Latchkey.

CI health: A - excellent

Point runs-on at Latchkey and get job timeouts, self-healing for flaky steps, and up to 58% lower cost, applied automatically.

Grade your own workflow free or run it on Latchkey →

Source: scikit-hep/awkward.github/workflows/deploy.ymlLicense BSD-3-ClauseView source

What it does

This is the Deploy workflow from the scikit-hep/awkward repository, a real project running GitHub Actions. It is shown here with attribution under its BSD-3-Clause license.

Below, Latchkey shows a faster, safer version produced by its optimization engine.

The workflow

workflow (.yml)

name: Deploy

on:
  workflow_dispatch:
    inputs:
        publish-pypi:
            type: boolean
            description: Publish to PyPI
  release:
    types:
    - published

permissions:
  contents: read

concurrency:
  group: deploy-${{ github.ref }}
  cancel-in-progress: true

jobs:
  determine-source-date-epoch:
    name: "Determine SOURCE_DATE_EPOCH"
    runs-on: ubuntu-latest
    outputs:
      source-date-epoch: ${{ steps.log.outputs.source-date-epoch }}

    steps:
      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
        with:
          submodules: true
          fetch-depth: 0
          persist-credentials: false

      - id: log
        name: Compute SOURCE_DATE_EPOCH
        run: |
          # Find latest unix timestamp in awkward-cpp, and the kernel generation files
          epoch=$( git log -1 --format=%at -- awkward-cpp kernel-specification.yml kernel-test-data.json )
          echo "source-date-epoch=$epoch" >> $GITHUB_OUTPUT

  check-requirements:
    name: "Check awkward requirements"
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
        with:
          submodules: true
          persist-credentials: false

      - name: Check awkward-cpp version matches requirement
        run: pipx run nox -s check_cpp_constraint

  check-cpp-on-pypi:
    name: "Check awkward-cpp dependency on PyPI"
    runs-on: ubuntu-latest
    needs: [determine-source-date-epoch]
    env:
      SOURCE_DATE_EPOCH: ${{ needs.determine-source-date-epoch.outputs.source-date-epoch }}

    steps:
      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
        with:
          submodules: true
          persist-credentials: false

      - name: Prepare build files
        run: pipx run nox -s prepare

      - name: Build awkward-cpp sdist
        run: pipx run build --sdist awkward-cpp

      - name: Check sdist matches PyPI
        run: pipx run nox -s check_cpp_sdist_released -- awkward-cpp/dist/awkward_cpp*.tar.gz

  build:
    name: "Build wheel & sdist"
    runs-on: ubuntu-latest
    needs: [determine-source-date-epoch]
    permissions:
      id-token: write # Required to sign release artifacts before publishing.
      attestations: write # Required to upload artifact attestations.
      contents: read
    env:
      SOURCE_DATE_EPOCH: ${{ needs.determine-source-date-epoch.outputs.source-date-epoch }}
    steps:
    - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
      with:
        submodules: true
        persist-credentials: false

    - name: Prepare build files
      run: pipx run nox -s prepare

    - name: Build distributions
      run: pipx run build

    - name: Check metadata
      run: pipx run twine check dist/*

    - name: Generate artifact attestation for sdist and wheel
      uses: actions/attest-build-provenance@0f67c3f4856b2e3261c31976d6725780e5e4c373 # v4.1.1
      with:
        subject-path: "dist/awkward-*"

    - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
      with:
        name: distributions
        path: dist/*

  bundle-headers:
    name: "Bundle header-only libraries"
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
      with:
        persist-credentials: false
    - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
      with:
        name: header-only
        path: header-only

  upload:
    name: Upload distributions
    needs: [build, check-requirements, check-cpp-on-pypi]
    runs-on: ubuntu-latest
    if: (github.event_name == 'release' && github.event.action == 'published') || inputs.publish-pypi
    permissions:
      id-token: write # Required for trusted publishing to PyPI.
      contents: read
    environment:
      name: "pypi"
      url: "https://pypi.org/project/awkward/"
    steps:
    - uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
      with:
        name: distributions
        path: dist

    - name: List distributions to be deployed
      run: ls -l dist/

    - name: Verify sdist artifact attestation
      env:
        GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        REPOSITORY: ${{ github.repository }}
      run: gh attestation verify dist/awkward-*.tar.gz --repo "${REPOSITORY}"

    - name: Verify wheel artifact attestation
      env:
        GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        REPOSITORY: ${{ github.repository }}
      run: gh attestation verify dist/awkward-*.whl --repo "${REPOSITORY}"

    - uses: pypa/gh-action-pypi-publish@cef221092ed1bacb1cc03d23a2d87d1d172e277b # v1.14.0

  publish-headers:
    name: "Publish header-only libraries alongside release"
    runs-on: ubuntu-latest
    needs: [bundle-headers]
    if: github.event_name == 'release' && github.event.action == 'published'
    permissions:
      contents: write # Required to upload header-only archives to releases.
    steps:
    - uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
      with:
        name: header-only
        path: header-only
    - name: Create archive
      run: |
        # Don't include `header-only` parent directory
        env -C header-only/ zip -r header-only.zip .
    - name: Upload archive to release
      env:
        GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        RELEASE_TAG: ${{ github.ref_name }}
        REPOSITORY: ${{ github.repository }}
      run: gh release upload "${RELEASE_TAG}" header-only/header-only.zip --repo "${REPOSITORY}" --clobber

The same workflow, on Latchkey

Removes redundant runs and caps runaway jobs. Added and changed lines are highlighted.

name: Deploy
 
on:
  workflow_dispatch:
    inputs:
        publish-pypi:
            type: boolean
            description: Publish to PyPI
  release:
    types:
    - published
 
permissions:
  contents: read
 
concurrency:
  group: deploy-${{ github.ref }}
  cancel-in-progress: true
 
jobs:
  determine-source-date-epoch:
    timeout-minutes: 30
    name: "Determine SOURCE_DATE_EPOCH"
    runs-on: latchkey-small
    outputs:
      source-date-epoch: ${{ steps.log.outputs.source-date-epoch }}
 
    steps:
      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
        with:
          submodules: true
          fetch-depth: 0
          persist-credentials: false
 
      - id: log
        name: Compute SOURCE_DATE_EPOCH
        run: |
          # Find latest unix timestamp in awkward-cpp, and the kernel generation files
          epoch=$( git log -1 --format=%at -- awkward-cpp kernel-specification.yml kernel-test-data.json )
          echo "source-date-epoch=$epoch" >> $GITHUB_OUTPUT
 
  check-requirements:
    timeout-minutes: 30
    name: "Check awkward requirements"
    runs-on: latchkey-small
 
    steps:
      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
        with:
          submodules: true
          persist-credentials: false
 
      - name: Check awkward-cpp version matches requirement
        run: pipx run nox -s check_cpp_constraint
 
  check-cpp-on-pypi:
    timeout-minutes: 30
    name: "Check awkward-cpp dependency on PyPI"
    runs-on: latchkey-small
    needs: [determine-source-date-epoch]
    env:
      SOURCE_DATE_EPOCH: ${{ needs.determine-source-date-epoch.outputs.source-date-epoch }}
 
    steps:
      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
        with:
          submodules: true
          persist-credentials: false
 
      - name: Prepare build files
        run: pipx run nox -s prepare
 
      - name: Build awkward-cpp sdist
        run: pipx run build --sdist awkward-cpp
 
      - name: Check sdist matches PyPI
        run: pipx run nox -s check_cpp_sdist_released -- awkward-cpp/dist/awkward_cpp*.tar.gz
 
  build:
    timeout-minutes: 30
    name: "Build wheel & sdist"
    runs-on: latchkey-small
    needs: [determine-source-date-epoch]
    permissions:
      id-token: write # Required to sign release artifacts before publishing.
      attestations: write # Required to upload artifact attestations.
      contents: read
    env:
      SOURCE_DATE_EPOCH: ${{ needs.determine-source-date-epoch.outputs.source-date-epoch }}
    steps:
    - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
      with:
        submodules: true
        persist-credentials: false
 
    - name: Prepare build files
      run: pipx run nox -s prepare
 
    - name: Build distributions
      run: pipx run build
 
    - name: Check metadata
      run: pipx run twine check dist/*
 
    - name: Generate artifact attestation for sdist and wheel
      uses: actions/attest-build-provenance@0f67c3f4856b2e3261c31976d6725780e5e4c373 # v4.1.1
      with:
        subject-path: "dist/awkward-*"
 
    - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
      with:
        name: distributions
        path: dist/*
 
  bundle-headers:
    timeout-minutes: 30
    name: "Bundle header-only libraries"
    runs-on: latchkey-small
    steps:
    - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
      with:
        persist-credentials: false
    - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
      with:
        name: header-only
        path: header-only
 
  upload:
    timeout-minutes: 30
    name: Upload distributions
    needs: [build, check-requirements, check-cpp-on-pypi]
    runs-on: latchkey-small
    if: (github.event_name == 'release' && github.event.action == 'published') || inputs.publish-pypi
    permissions:
      id-token: write # Required for trusted publishing to PyPI.
      contents: read
    environment:
      name: "pypi"
      url: "https://pypi.org/project/awkward/"
    steps:
    - uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
      with:
        name: distributions
        path: dist
 
    - name: List distributions to be deployed
      run: ls -l dist/
 
    - name: Verify sdist artifact attestation
      env:
        GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        REPOSITORY: ${{ github.repository }}
      run: gh attestation verify dist/awkward-*.tar.gz --repo "${REPOSITORY}"
 
    - name: Verify wheel artifact attestation
      env:
        GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        REPOSITORY: ${{ github.repository }}
      run: gh attestation verify dist/awkward-*.whl --repo "${REPOSITORY}"
 
    - uses: pypa/gh-action-pypi-publish@cef221092ed1bacb1cc03d23a2d87d1d172e277b # v1.14.0
 
  publish-headers:
    timeout-minutes: 30
    name: "Publish header-only libraries alongside release"
    runs-on: latchkey-small
    needs: [bundle-headers]
    if: github.event_name == 'release' && github.event.action == 'published'
    permissions:
      contents: write # Required to upload header-only archives to releases.
    steps:
    - uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
      with:
        name: header-only
        path: header-only
    - name: Create archive
      run: |
        # Don't include `header-only` parent directory
        env -C header-only/ zip -r header-only.zip .
    - name: Upload archive to release
      env:
        GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        RELEASE_TAG: ${{ github.ref_name }}
        REPOSITORY: ${{ github.repository }}
      run: gh release upload "${RELEASE_TAG}" header-only/header-only.zip --repo "${REPOSITORY}" --clobber

What changed

Run on Latchkey managed runners with one line (runs-on), which apply the fixes below automatically and self-heal transient failures. This example uses latchkey-small; pick the runner size that fits the job.
Add a job timeout so a hung step cannot burn hours of runner time.

This workflow runs 7 jobs per trigger. On Latchkey the same minutes cost up to 58% less than GitHub-hosted, with zero queue time.

Actions used in this workflow

actions/checkout actions/upload-artifact actions/download-artifact