linting-typechecking-doctests workflow (pyranges/pyranges0)

The linting-typechecking-doctests workflow from pyranges/pyranges0, explained and optimized by Latchkey.

CI health: D - needs work

Point runs-on at Latchkey and get caching, run de-duplication, job timeouts, self-healing for flaky steps, and up to 58% lower cost, applied automatically.

Grade your own workflow free or run it on Latchkey →

Source: pyranges/pyranges0.github/workflows/install_doctest_lint_typecheck.ymlLicense MITView source

What it does

This is the linting-typechecking-doctests workflow from the pyranges/pyranges0 repository, a real project running GitHub Actions. It is shown here with attribution under its MIT license.

Below, Latchkey shows a faster, safer version produced by its optimization engine.

The workflow

workflow (.yml)

name: linting-typechecking-doctests

# Source: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python#using-multiple-python-versions

on: [pull_request, workflow_dispatch]

jobs:
  install_and_test:
    runs-on: ubuntu-latest
    strategy:
      # You can use PyPy versions in python-version.
      # For example, pypy2.7 and pypy3.9
      matrix:
        python-version: ["3.7", "3.8", "3.9", "3.10", "3.11"]

    steps:
      - uses: actions/checkout@v3
      - name: Set up Python ${{ matrix.python-version }}
        uses: actions/setup-python@v4
        with:
          python-version: ${{ matrix.python-version }}
      - name: Install
        run: pip install .
      - name: Install add-ons
        run: pip install pyrle pybigwig bamread pyfaidx fisher
      - run: pip install pytest
      - name: doctests
        env:
          GITHUB_ACTIONS: true
        run: |
          shopt -s globstar  # allow recursive globs
          python -m doctest pyranges/**.py
      - name: unittests
        env:
          GITHUB_ACTIONS: true
        run: |
          pytest tests/unit

  isort:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Set up Python environment
        uses: actions/setup-python@v4
        with:
          python-version: "3.11"
      - run: pip install isort
      - run: isort --profile black -l 120 --check --diff .

  black:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Set up Python environment
        uses: actions/setup-python@v4
        with:
          python-version: "3.11.0"
          max-line-length: "120"
      - run: pip install black
      - run: black -l 120 --check --diff pyranges tests

  flake8:
    runs-on: ubuntu-latest
    name: Lint
    steps:
      - name: Check out source repository
        uses: actions/checkout@v3
      - name: Set up Python environment
        uses: actions/setup-python@v4
      - name: flake8 Lint install
        run: pip install flake8
      - name: flake8 Lint
        run: flake8 --max-line-length=120 --ignore E203,E501,W503 pyranges

  mypy:
    runs-on: ubuntu-latest
    name: Mypy
    steps:
      - name: Check out source repository
        uses: actions/checkout@v3
      - name: Set up Python environment
        uses: actions/setup-python@v4
        with:
          python-version: "3.11"
      - name: Install mypy
        run: pip install mypy
      - name: mypy
        run: |
          pip install numpy  # for the typing stubs
          python -m pip install pandas-stubs types-setuptools types-tabulate 
          mypy pyranges

The same workflow, on Latchkey

Estimated ~20% faster on cache hits, plus fewer wasted runs and a safer supply chain. Added and changed lines are highlighted.

name: linting-typechecking-doctests
 
# Source: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python#using-multiple-python-versions
 
on: [pull_request, workflow_dispatch]
 
concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true
 
jobs:
  install_and_test:
    timeout-minutes: 30
    runs-on: latchkey-small
    strategy:
      # You can use PyPy versions in python-version.
      # For example, pypy2.7 and pypy3.9
      matrix:
        python-version: ["3.7", "3.8", "3.9", "3.10", "3.11"]
 
    steps:
      - uses: actions/checkout@v3
      - name: Set up Python ${{ matrix.python-version }}
        uses: actions/setup-python@v4
        with:
          cache: 'pip'
          python-version: ${{ matrix.python-version }}
      - name: Install
        run: pip install .
      - name: Install add-ons
        run: pip install pyrle pybigwig bamread pyfaidx fisher
      - run: pip install pytest
      - name: doctests
        env:
          GITHUB_ACTIONS: true
        run: |
          shopt -s globstar  # allow recursive globs
          python -m doctest pyranges/**.py
      - name: unittests
        env:
          GITHUB_ACTIONS: true
        run: |
          pytest tests/unit
 
  isort:
    timeout-minutes: 30
    runs-on: latchkey-small
    steps:
      - uses: actions/checkout@v3
      - name: Set up Python environment
        uses: actions/setup-python@v4
        with:
          cache: 'pip'
          python-version: "3.11"
      - run: pip install isort
      - run: isort --profile black -l 120 --check --diff .
 
  black:
    timeout-minutes: 30
    runs-on: latchkey-small
    steps:
      - uses: actions/checkout@v3
      - name: Set up Python environment
        uses: actions/setup-python@v4
        with:
          cache: 'pip'
          python-version: "3.11.0"
          max-line-length: "120"
      - run: pip install black
      - run: black -l 120 --check --diff pyranges tests
 
  flake8:
    timeout-minutes: 30
    runs-on: latchkey-small
    name: Lint
    steps:
      - name: Check out source repository
        uses: actions/checkout@v3
      - name: Set up Python environment
        uses: actions/setup-python@v4
        with:
          cache: 'pip'
      - name: flake8 Lint install
        run: pip install flake8
      - name: flake8 Lint
        run: flake8 --max-line-length=120 --ignore E203,E501,W503 pyranges
 
  mypy:
    timeout-minutes: 30
    runs-on: latchkey-small
    name: Mypy
    steps:
      - name: Check out source repository
        uses: actions/checkout@v3
      - name: Set up Python environment
        uses: actions/setup-python@v4
        with:
          cache: 'pip'
          python-version: "3.11"
      - name: Install mypy
        run: pip install mypy
      - name: mypy
        run: |
          pip install numpy  # for the typing stubs
          python -m pip install pandas-stubs types-setuptools types-tabulate 
          mypy pyranges

What changed

Run on Latchkey managed runners with one line (runs-on), which apply the fixes below automatically and self-heal transient failures. This example uses latchkey-small; pick the runner size that fits the job.
Cancel superseded runs when a branch or PR gets a newer push.
Cache dependency installs on the setup step so they are served from cache.
Add a job timeout so a hung step cannot burn hours of runner time.

What Latchkey heals here

This workflow has steps that commonly fail on transient issues (network, registries, flaky browsers). On Latchkey managed runners they are detected, retried, and self-healed instead of failing your build:

Dependency installs

This workflow runs 5 jobs (9 with the matrix expanded) per trigger. On Latchkey the same minutes cost up to 58% less than GitHub-hosted, with zero queue time.

Actions used in this workflow

actions/checkout actions/setup-python