Test workflow (jackmpcollins/magentic)

The Test workflow from jackmpcollins/magentic, explained and optimized by Latchkey.

CI health: F - at risk

Point runs-on at Latchkey and get caching, run de-duplication, job timeouts, SHA-pinned actions, self-healing for flaky steps, and up to 58% lower cost, applied automatically.

Grade your own workflow free or run it on Latchkey →

Source: jackmpcollins/magentic.github/workflows/test.ymlLicense MITView source

What it does

This is the Test workflow from the jackmpcollins/magentic repository, a real project running GitHub Actions. It is shown here with attribution under its MIT license.

Below, Latchkey shows a faster, safer version produced by its optimization engine.

The workflow

workflow (.yml)

name: Test

on:
  pull_request:
  push:
    branches:
      - main
  workflow_dispatch:
    inputs:
      # Options to test specific APIs ignoring the VCR cassettes
      anthropic:
        description: "Test using Anthropic API"
        type: boolean
        default: false
      litellm_anthropic:
        description: "Test using Anthropic API via LiteLLM"
        type: boolean
        default: false
      litellm_openai:
        description: "Test using OpenAI API via LiteLLM"
        type: boolean
        default: false
      mistral:
        description: "Test using Mistral API"
        type: boolean
        default: false
      openai:
        description: "Test using OpenAI API"
        type: boolean
        default: false

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v5
      - name: Set up Python
        uses: actions/setup-python@v6
      - name: Set up uv
        uses: astral-sh/setup-uv@v7
      - name: Install dependencies
        run: uv sync
      - run: uv run ruff check --output-format=github .
      - run: uv run ruff format --check .
      - run: uv run mypy .
      - run: uv run pytest -vv tests/
      # Do not use `github.event.inputs` because it converts booleans to strings
      # https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#onworkflow_dispatchinputs
      - if: inputs.anthropic
        env:
          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
        run: uv run pytest -vv tests/ -m 'anthropic' --record-mode=rewrite
      - if: inputs.litellm_anthropic
        env:
          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
        run: uv run pytest -vv tests/ -m 'litellm_anthropic' --record-mode=rewrite
      - if: inputs.litellm_openai
        env:
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
          OPENAI_ORG_ID: ${{ secrets.OPENAI_ORG_ID }}
        run: uv run pytest -vv tests/ -m 'litellm_openai' --record-mode=rewrite
      - if: inputs.mistral
        env:
          MISTRAL_API_KEY: ${{ secrets.MISTRAL_API_KEY }}
        run: uv run pytest -vv tests/ -m 'mistral' --record-mode=rewrite
      - if: inputs.openai
        env:
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
          OPENAI_ORG_ID: ${{ secrets.OPENAI_ORG_ID }}
        run: uv run pytest -vv tests/ -m 'openai' --record-mode=rewrite

The same workflow, on Latchkey

Estimated ~20% faster on cache hits, plus fewer wasted runs and a safer supply chain. Added and changed lines are highlighted.

name: Test
 
on:
  pull_request:
  push:
    branches:
      - main
  workflow_dispatch:
    inputs:
      # Options to test specific APIs ignoring the VCR cassettes
      anthropic:
        description: "Test using Anthropic API"
        type: boolean
        default: false
      litellm_anthropic:
        description: "Test using Anthropic API via LiteLLM"
        type: boolean
        default: false
      litellm_openai:
        description: "Test using OpenAI API via LiteLLM"
        type: boolean
        default: false
      mistral:
        description: "Test using Mistral API"
        type: boolean
        default: false
      openai:
        description: "Test using OpenAI API"
        type: boolean
        default: false
 
concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true
 
jobs:
  test:
    timeout-minutes: 30
    runs-on: latchkey-small
    steps:
      - uses: actions/checkout@v5
      - name: Set up Python
        uses: actions/setup-python@v6
        with:
          cache: 'pip'
      - name: Set up uv
        uses: astral-sh/setup-uv@v7
      - name: Install dependencies
        run: uv sync
      - run: uv run ruff check --output-format=github .
      - run: uv run ruff format --check .
      - run: uv run mypy .
      - run: uv run pytest -vv tests/
      # Do not use `github.event.inputs` because it converts booleans to strings
      # https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#onworkflow_dispatchinputs
      - if: inputs.anthropic
        env:
          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
        run: uv run pytest -vv tests/ -m 'anthropic' --record-mode=rewrite
      - if: inputs.litellm_anthropic
        env:
          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
        run: uv run pytest -vv tests/ -m 'litellm_anthropic' --record-mode=rewrite
      - if: inputs.litellm_openai
        env:
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
          OPENAI_ORG_ID: ${{ secrets.OPENAI_ORG_ID }}
        run: uv run pytest -vv tests/ -m 'litellm_openai' --record-mode=rewrite
      - if: inputs.mistral
        env:
          MISTRAL_API_KEY: ${{ secrets.MISTRAL_API_KEY }}
        run: uv run pytest -vv tests/ -m 'mistral' --record-mode=rewrite
      - if: inputs.openai
        env:
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
          OPENAI_ORG_ID: ${{ secrets.OPENAI_ORG_ID }}
        run: uv run pytest -vv tests/ -m 'openai' --record-mode=rewrite

What changed

Run on Latchkey managed runners with one line (runs-on), which apply the fixes below automatically and self-heal transient failures. This example uses latchkey-small; pick the runner size that fits the job.
Cancel superseded runs when a branch or PR gets a newer push.
Cache dependency installs on the setup step so they are served from cache.
Add a job timeout so a hung step cannot burn hours of runner time.

1 third-party action is referenced by a movable tag. Pin it to the commit SHA (Latchkey resolves and applies this automatically) so a repointed tag cannot change what runs.

This workflow runs 1 job per trigger. On Latchkey the same minutes cost up to 58% less than GitHub-hosted, with zero queue time.

Actions used in this workflow

actions/checkout actions/setup-python astral-sh/setup-uv