Skip to content
Latchkey

CI workflow (ProjectQ-Framework/ProjectQ)

The CI workflow from ProjectQ-Framework/ProjectQ, explained and optimized by Latchkey.

A

CI health: A - excellent

Point runs-on at Latchkey and get job timeouts, self-healing for flaky steps, and up to 58% lower cost, applied automatically.

Grade your own workflow free or run it on Latchkey →
Source: ProjectQ-Framework/ProjectQ.github/workflows/ci.ymlLicense Apache-2.0View source

What it does

This is the CI workflow from the ProjectQ-Framework/ProjectQ repository, a real project running GitHub Actions. It is shown here with attribution under its Apache-2.0 license.

Below, Latchkey shows a faster, safer version produced by its optimization engine.

The workflow

workflow (.yml)
---

name: CI

on:
  workflow_dispatch:
  merge_group:
  pull_request:
  push:
    branches:
      - master
      - develop
      - v*

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

jobs:
  standard:
    strategy:
      fail-fast: false
      matrix:
        runs-on: [ubuntu-latest, windows-latest, macos-latest]
        python:
          - 3.8
          - 3.9
          - '3.10'
          - '3.11'
          - '3.12'

    name: "Python ${{ matrix.python }} • ${{ matrix.runs-on }} • x64 ${{ matrix.args }}"
    runs-on: ${{ matrix.runs-on }}

    steps:
      - uses: actions/checkout@v3

      - name: Get history and tags for SCM versioning to work
        if: ${{ !env.ACT }}
        run: |
          git fetch --prune --unshallow
          git fetch --depth=1 origin +refs/tags/*:refs/tags/*

      - name: Setup Python ${{ matrix.python }}
        uses: actions/setup-python@v4
        with:
          python-version: ${{ matrix.python }}
          architecture: 'x64'
          cache: 'pip'
          cache-dependency-path: |
            setup.cfg
            pyproject.toml

      - name: Prepare env
        run: |
          python -m pip install -U "setuptools>=61.0.0"

      - name: Generate requirement file (Unix)
        if: runner.os != 'Windows'
        run: |
          python setup.py gen_reqfile --include-extras=test,azure-quantum,braket,revkit

      - name: Generate requirement file (Windows)
        if: runner.os == 'Windows'
        run: |
          python setup.py gen_reqfile --include-extras=test,azure-quantum,braket

      - name: Prepare env
        run: |
          python -m pip install -U pip setuptools wheel pybind11
          cat requirements.txt
          python -m pip install -r requirements.txt --prefer-binary
          python -m pip install coveralls

      - name: Setup annotations on Linux
        if: runner.os == 'Linux'
        run: python -m pip install pytest-github-actions-annotate-failures

      - name: Build and install package (Unix)
        if: runner.os != 'Windows'
        run: python -m pip install -ve .[azure-quantum,braket,revkit,test]

      - name: Build and install package (Windows)
        if: runner.os == 'Windows'
        run: python -m pip install -ve .[azure-quantum,braket,test]

      - name: Pytest
        run: |
          echo 'backend: Agg' > matplotlibrc
          python -m pytest -p no:warnings --cov=projectq

      - name: Coveralls.io
        run: coveralls --service=github
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          COVERALLS_FLAG_NAME: python-${{ matrix.python }}-${{ matrix.runs-on }}-x64
          COVERALLS_PARALLEL: true

  finish:
    needs: standard
    runs-on: ubuntu-latest
    container: python:3-slim
    steps:
      - name: Coveralls Finished
        run: |
          pip3 install --upgrade coveralls
          coveralls --finish
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}


  clang:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        clang:
          - 11
          - latest
    env:
      CC: clang
      CXX: clang++
      PROJECTQ_CLEANUP_COMPILER_FLAGS: ${{ (matrix.clang <= 10) && 1 || 0 }}

    name: "Python 3 • Clang ${{ matrix.clang }} • x64"
    container: "silkeh/clang:${{ matrix.clang }}"

    steps:
      - name: Install Git
        run: |
          apt-get update && apt-get install -y git --no-install-recommends

      # Work-around for https://github.com/actions/runner-images/issues/6775
      - name: Change Owner of Container Working Directory
        run: chown root:root .

      - uses: actions/checkout@v3

      - name: Get history and tags for SCM versioning to work
        if: ${{ !env.ACT }}
        run: |
          git fetch --prune --unshallow
          git fetch --depth=1 origin +refs/tags/*:refs/tags/*

      - name: Prepare env
        run: >
          apt-get update && apt-get install -y python3-dev python3-pip python3-setuptools python3-wheel
          python3-numpy python3-scipy python3-matplotlib python3-requests python3-networkx
          python3-pytest python3-pytest-cov python3-flaky python3-venv
          --no-install-recommends

      - name: Prepare Python env
        run: |
          python3 -m venv venv
          ./venv/bin/python3 -m pip install -U pip setuptools wheel
          ./venv/bin/python3 setup.py gen_reqfile --include-extras=test,azure-quantum,braket
          cat requirements.txt
          ./venv/bin/python3 -m pip install -r requirements.txt --prefer-binary

      - name: Upgrade pybind11 and flaky
        run: ./venv/bin/python3 -m pip install --upgrade pybind11 flaky --prefer-binary

      - name: Build and install package
        run: ./venv/bin/python3 -m pip install -ve .[azure-quantum,braket,test]

      - name: Pytest
        run: |
          echo 'backend: Agg' > matplotlibrc
          ./venv/bin/python3 -m pytest -p no:warnings


  gcc:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        gcc:
          - 9
          - latest

    name: "Python 3 • GCC ${{ matrix.gcc }} • x64"
    container: "gcc:${{ matrix.gcc }}"

    steps:
      - uses: actions/checkout@v3

      # Work-around for https://github.com/actions/runner-images/issues/6775
      - name: Change Owner of Container Working Directory
        run: chown root:root .

      - name: Get history and tags for SCM versioning to work
        if: ${{ !env.ACT }}
        run: |
          git fetch --prune --unshallow
          git fetch --depth=1 origin +refs/tags/*:refs/tags/*

      - name: Prepare env
        run: >
          apt-get update && apt-get install -y python3-dev python3-pip python3-setuptools python3-wheel
          python3-numpy python3-scipy python3-matplotlib python3-requests python3-networkx
          python3-pytest python3-pytest-cov python3-flaky python3-venv
          --no-install-recommends

      - name: Prepare Python env
        run: |
          python3 -m venv venv
          ./venv/bin/python3 -m pip install -U pip setuptools wheel
          ./venv/bin/python3 setup.py gen_reqfile --include-extras=test,azure-quantum,braket
          cat requirements.txt
          ./venv/bin/python3 -m pip install -r requirements.txt --prefer-binary

      - name: Upgrade pybind11 and flaky
        run: ./venv/bin/python3 -m pip install --upgrade pybind11 flaky --prefer-binary

      - name: Build and install package
        run: ./venv/bin/python3 -m pip install -ve .[azure-quantum,braket,test]

      - name: Pytest
        run: |
          echo 'backend: Agg' > matplotlibrc
          ./venv/bin/python3 -m pytest -p no:warnings


  # Testing on CentOS (manylinux uses a centos base, and this is an easy way
  # to get GCC 4.8, which is the manylinux1 compiler).
  centos:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        centos:
          - 7  # GCC 4.8
          - 8
        include:
          - centos: 7
            python_pkg: rh-python38-python-pip rh-python38-python-devel
            enable_repo: --enablerepo=centos-sclo-rh
          - centos: 8
            python_pkg: python38-devel


    name: "Python 3 • CentOS ${{ matrix.centos }} • x64"
    container: "centos:${{ matrix.centos }}"

    steps:
      - name: Enable cache for yum
        run: echo 'keepcache=1' >> /etc/yum.conf

      - name: Setup yum cache
        uses: actions/cache@v3
        with:
          path: |
            /var/cache/yum/
            /var/cache/dnf/
          key: ${{ runner.os }}-centos${{ matrix.centos }}-yum-${{ secrets.yum_cache }}

      - name: Fix repository URLs (CentOS 8 only)
        if: matrix.centos == 8
        run: |
          sed -i 's/mirrorlist/#mirrorlist/g' /etc/yum.repos.d/CentOS-*
          sed -i 's|#baseurl=http://mirror.centos.org|baseurl=http://vault.centos.org|g' /etc/yum.repos.d/CentOS-*

      - name: Update YUM/DNF
        run: yum update -y

      - name: Enable extra repositories && modify PATH (CentOS 7)
        if: matrix.centos == 7
        run: |
          yum install -y centos-release-scl-rh
          yum -y install https://packages.endpointdev.com/rhel/7/os/x86_64/endpoint-repo.x86_64.rpm
          echo '/opt/rh/rh-python38/root/usr/bin' >> $GITHUB_PATH

      - name: Add Python 3 and other dependencies
        run: yum install -y ${{ matrix.enable_repo }} ${{ matrix.python_pkg }} gcc-c++ make

      - name: Install Git > 2.18
        run: |
          yum install -y git
          git config --global --add safe.directory /__w/ProjectQ/ProjectQ

      # Work-around for https://github.com/actions/runner-images/issues/6775
      - name: Change Owner of Container Working Directory
        run: chown root:root .

      - uses: actions/checkout@v3

      - name: Get history and tags for SCM versioning to work
        if: ${{ !env.ACT }}
        run: |
          git fetch --prune --unshallow
          git fetch --depth=1 origin +refs/tags/*:refs/tags/*

      - name: Create pip cache dir
        run: mkdir -p ~/.cache/pip

      - name: Cache wheels
        uses: actions/cache@v3
        with:
          path: ~/.cache/pip
          key: ${{ runner.os }}-centos${{ matrix.centos }}-pip-${{ hashFiles('**/setup.cfg', '**/pyproject.toml') }}
          restore-keys: ${{ runner.os }}-centos-pip-

      - name: Install dependencies
        run: |
          python3 -m pip install -U pip setuptools wheel
          python3 setup.py gen_reqfile --include-extras=test,azure-quantum,braket
          cat requirements.txt
          python3 -m pip install -r requirements.txt --prefer-binary

      - name: Build and install package
        run: python3 -m pip install -ve .[azure-quantum,braket,test]

      - name: Pytest
        run: |
          echo 'backend: Agg' > matplotlibrc
          python3 -m pytest -p no:warnings


  documentation:
    name: "Documentation build test"
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v3

      - name: Get history and tags for SCM versioning to work
        if: ${{ !env.ACT }}
        run: |
          git fetch --prune --unshallow
          git fetch --depth=1 origin +refs/tags/*:refs/tags/*

      - uses: actions/setup-python@v4
        with:
          python-version: '3.x'
          architecture: 'x64'
          cache: 'pip'
          cache-dependency-path: |
            setup.cfg
            pyproject.toml

      - name: Install docs & setup requirements
        run: |
          python3 -m pip install .[docs]

      - name: Build docs
        run: python3 -m sphinx -b html docs docs/.build

      # NB: disabling until setup.py is updated to remove any mention of distutils
      # - name: Make SDist
      #  run: python3 setup.py sdist

The same workflow, on Latchkey

Removes redundant runs and caps runaway jobs. Added and changed lines are highlighted.

---
 
name: CI
 
on:
  workflow_dispatch:
  merge_group:
  pull_request:
  push:
    branches:
      - master
      - develop
      - v*
 
concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true
 
jobs:
  standard:
    timeout-minutes: 30
    strategy:
      fail-fast: false
      matrix:
        runs-on: [ubuntu-latest, windows-latest, macos-latest]
        python:
          - 3.8
          - 3.9
          - '3.10'
          - '3.11'
          - '3.12'
 
    name: "Python ${{ matrix.python }} • ${{ matrix.runs-on }} • x64 ${{ matrix.args }}"
    runs-on: ${{ matrix.runs-on }}
 
    steps:
      - uses: actions/checkout@v3
 
      - name: Get history and tags for SCM versioning to work
        if: ${{ !env.ACT }}
        run: |
          git fetch --prune --unshallow
          git fetch --depth=1 origin +refs/tags/*:refs/tags/*
 
      - name: Setup Python ${{ matrix.python }}
        uses: actions/setup-python@v4
        with:
          python-version: ${{ matrix.python }}
          architecture: 'x64'
          cache: 'pip'
          cache-dependency-path: |
            setup.cfg
            pyproject.toml
 
      - name: Prepare env
        run: |
          python -m pip install -U "setuptools>=61.0.0"
 
      - name: Generate requirement file (Unix)
        if: runner.os != 'Windows'
        run: |
          python setup.py gen_reqfile --include-extras=test,azure-quantum,braket,revkit
 
      - name: Generate requirement file (Windows)
        if: runner.os == 'Windows'
        run: |
          python setup.py gen_reqfile --include-extras=test,azure-quantum,braket
 
      - name: Prepare env
        run: |
          python -m pip install -U pip setuptools wheel pybind11
          cat requirements.txt
          python -m pip install -r requirements.txt --prefer-binary
          python -m pip install coveralls
 
      - name: Setup annotations on Linux
        if: runner.os == 'Linux'
        run: python -m pip install pytest-github-actions-annotate-failures
 
      - name: Build and install package (Unix)
        if: runner.os != 'Windows'
        run: python -m pip install -ve .[azure-quantum,braket,revkit,test]
 
      - name: Build and install package (Windows)
        if: runner.os == 'Windows'
        run: python -m pip install -ve .[azure-quantum,braket,test]
 
      - name: Pytest
        run: |
          echo 'backend: Agg' > matplotlibrc
          python -m pytest -p no:warnings --cov=projectq
 
      - name: Coveralls.io
        run: coveralls --service=github
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          COVERALLS_FLAG_NAME: python-${{ matrix.python }}-${{ matrix.runs-on }}-x64
          COVERALLS_PARALLEL: true
 
  finish:
    timeout-minutes: 30
    needs: standard
    runs-on: latchkey-small
    container: python:3-slim
    steps:
      - name: Coveralls Finished
        run: |
          pip3 install --upgrade coveralls
          coveralls --finish
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
 
 
  clang:
    timeout-minutes: 30
    runs-on: latchkey-small
    strategy:
      fail-fast: false
      matrix:
        clang:
          - 11
          - latest
    env:
      CC: clang
      CXX: clang++
      PROJECTQ_CLEANUP_COMPILER_FLAGS: ${{ (matrix.clang <= 10) && 1 || 0 }}
 
    name: "Python 3 • Clang ${{ matrix.clang }} • x64"
    container: "silkeh/clang:${{ matrix.clang }}"
 
    steps:
      - name: Install Git
        run: |
          apt-get update && apt-get install -y git --no-install-recommends
 
      # Work-around for https://github.com/actions/runner-images/issues/6775
      - name: Change Owner of Container Working Directory
        run: chown root:root .
 
      - uses: actions/checkout@v3
 
      - name: Get history and tags for SCM versioning to work
        if: ${{ !env.ACT }}
        run: |
          git fetch --prune --unshallow
          git fetch --depth=1 origin +refs/tags/*:refs/tags/*
 
      - name: Prepare env
        run: >
          apt-get update && apt-get install -y python3-dev python3-pip python3-setuptools python3-wheel
          python3-numpy python3-scipy python3-matplotlib python3-requests python3-networkx
          python3-pytest python3-pytest-cov python3-flaky python3-venv
          --no-install-recommends
 
      - name: Prepare Python env
        run: |
          python3 -m venv venv
          ./venv/bin/python3 -m pip install -U pip setuptools wheel
          ./venv/bin/python3 setup.py gen_reqfile --include-extras=test,azure-quantum,braket
          cat requirements.txt
          ./venv/bin/python3 -m pip install -r requirements.txt --prefer-binary
 
      - name: Upgrade pybind11 and flaky
        run: ./venv/bin/python3 -m pip install --upgrade pybind11 flaky --prefer-binary
 
      - name: Build and install package
        run: ./venv/bin/python3 -m pip install -ve .[azure-quantum,braket,test]
 
      - name: Pytest
        run: |
          echo 'backend: Agg' > matplotlibrc
          ./venv/bin/python3 -m pytest -p no:warnings
 
 
  gcc:
    timeout-minutes: 30
    runs-on: latchkey-small
    strategy:
      fail-fast: false
      matrix:
        gcc:
          - 9
          - latest
 
    name: "Python 3 • GCC ${{ matrix.gcc }} • x64"
    container: "gcc:${{ matrix.gcc }}"
 
    steps:
      - uses: actions/checkout@v3
 
      # Work-around for https://github.com/actions/runner-images/issues/6775
      - name: Change Owner of Container Working Directory
        run: chown root:root .
 
      - name: Get history and tags for SCM versioning to work
        if: ${{ !env.ACT }}
        run: |
          git fetch --prune --unshallow
          git fetch --depth=1 origin +refs/tags/*:refs/tags/*
 
      - name: Prepare env
        run: >
          apt-get update && apt-get install -y python3-dev python3-pip python3-setuptools python3-wheel
          python3-numpy python3-scipy python3-matplotlib python3-requests python3-networkx
          python3-pytest python3-pytest-cov python3-flaky python3-venv
          --no-install-recommends
 
      - name: Prepare Python env
        run: |
          python3 -m venv venv
          ./venv/bin/python3 -m pip install -U pip setuptools wheel
          ./venv/bin/python3 setup.py gen_reqfile --include-extras=test,azure-quantum,braket
          cat requirements.txt
          ./venv/bin/python3 -m pip install -r requirements.txt --prefer-binary
 
      - name: Upgrade pybind11 and flaky
        run: ./venv/bin/python3 -m pip install --upgrade pybind11 flaky --prefer-binary
 
      - name: Build and install package
        run: ./venv/bin/python3 -m pip install -ve .[azure-quantum,braket,test]
 
      - name: Pytest
        run: |
          echo 'backend: Agg' > matplotlibrc
          ./venv/bin/python3 -m pytest -p no:warnings
 
 
  # Testing on CentOS (manylinux uses a centos base, and this is an easy way
  # to get GCC 4.8, which is the manylinux1 compiler).
  centos:
    timeout-minutes: 30
    runs-on: latchkey-small
    strategy:
      fail-fast: false
      matrix:
        centos:
          - 7  # GCC 4.8
          - 8
        include:
          - centos: 7
            python_pkg: rh-python38-python-pip rh-python38-python-devel
            enable_repo: --enablerepo=centos-sclo-rh
          - centos: 8
            python_pkg: python38-devel
 
 
    name: "Python 3 • CentOS ${{ matrix.centos }} • x64"
    container: "centos:${{ matrix.centos }}"
 
    steps:
      - name: Enable cache for yum
        run: echo 'keepcache=1' >> /etc/yum.conf
 
      - name: Setup yum cache
        uses: actions/cache@v3
        with:
          path: |
            /var/cache/yum/
            /var/cache/dnf/
          key: ${{ runner.os }}-centos${{ matrix.centos }}-yum-${{ secrets.yum_cache }}
 
      - name: Fix repository URLs (CentOS 8 only)
        if: matrix.centos == 8
        run: |
          sed -i 's/mirrorlist/#mirrorlist/g' /etc/yum.repos.d/CentOS-*
          sed -i 's|#baseurl=http://mirror.centos.org|baseurl=http://vault.centos.org|g' /etc/yum.repos.d/CentOS-*
 
      - name: Update YUM/DNF
        run: yum update -y
 
      - name: Enable extra repositories && modify PATH (CentOS 7)
        if: matrix.centos == 7
        run: |
          yum install -y centos-release-scl-rh
          yum -y install https://packages.endpointdev.com/rhel/7/os/x86_64/endpoint-repo.x86_64.rpm
          echo '/opt/rh/rh-python38/root/usr/bin' >> $GITHUB_PATH
 
      - name: Add Python 3 and other dependencies
        run: yum install -y ${{ matrix.enable_repo }} ${{ matrix.python_pkg }} gcc-c++ make
 
      - name: Install Git > 2.18
        run: |
          yum install -y git
          git config --global --add safe.directory /__w/ProjectQ/ProjectQ
 
      # Work-around for https://github.com/actions/runner-images/issues/6775
      - name: Change Owner of Container Working Directory
        run: chown root:root .
 
      - uses: actions/checkout@v3
 
      - name: Get history and tags for SCM versioning to work
        if: ${{ !env.ACT }}
        run: |
          git fetch --prune --unshallow
          git fetch --depth=1 origin +refs/tags/*:refs/tags/*
 
      - name: Create pip cache dir
        run: mkdir -p ~/.cache/pip
 
      - name: Cache wheels
        uses: actions/cache@v3
        with:
          path: ~/.cache/pip
          key: ${{ runner.os }}-centos${{ matrix.centos }}-pip-${{ hashFiles('**/setup.cfg', '**/pyproject.toml') }}
          restore-keys: ${{ runner.os }}-centos-pip-
 
      - name: Install dependencies
        run: |
          python3 -m pip install -U pip setuptools wheel
          python3 setup.py gen_reqfile --include-extras=test,azure-quantum,braket
          cat requirements.txt
          python3 -m pip install -r requirements.txt --prefer-binary
 
      - name: Build and install package
        run: python3 -m pip install -ve .[azure-quantum,braket,test]
 
      - name: Pytest
        run: |
          echo 'backend: Agg' > matplotlibrc
          python3 -m pytest -p no:warnings
 
 
  documentation:
    timeout-minutes: 30
    name: "Documentation build test"
    runs-on: latchkey-small
 
    steps:
      - uses: actions/checkout@v3
 
      - name: Get history and tags for SCM versioning to work
        if: ${{ !env.ACT }}
        run: |
          git fetch --prune --unshallow
          git fetch --depth=1 origin +refs/tags/*:refs/tags/*
 
      - uses: actions/setup-python@v4
        with:
          python-version: '3.x'
          architecture: 'x64'
          cache: 'pip'
          cache-dependency-path: |
            setup.cfg
            pyproject.toml
 
      - name: Install docs & setup requirements
        run: |
          python3 -m pip install .[docs]
 
      - name: Build docs
        run: python3 -m sphinx -b html docs docs/.build
 
      # NB: disabling until setup.py is updated to remove any mention of distutils
      # - name: Make SDist
      #  run: python3 setup.py sdist
 

What changed

What Latchkey heals here

This workflow has steps that commonly fail on transient issues (network, registries, flaky browsers). On Latchkey managed runners they are detected, retried, and self-healed instead of failing your build:

This workflow runs 6 jobs (23 with the matrix expanded) per trigger. On Latchkey the same minutes cost up to 58% less than GitHub-hosted, with zero queue time.

Actions used in this workflow