Tests workflow (GH05TCREW/pentestagent)

The Tests workflow from GH05TCREW/pentestagent, explained and optimized by Latchkey.

CI health: A - excellent

Point runs-on at Latchkey and get job timeouts, self-healing for flaky steps, and up to 58% lower cost, applied automatically.

Grade your own workflow free or run it on Latchkey →

Source: GH05TCREW/pentestagent.github/workflows/tests.ymlLicense MITView source

What it does

This is the Tests workflow from the GH05TCREW/pentestagent repository, a real project running GitHub Actions. It is shown here with attribution under its MIT license.

Below, Latchkey shows a faster, safer version produced by its optimization engine.

The workflow

workflow (.yml)

name: Tests

on:
  push:
    branches: ["**"]
  pull_request:
    branches: [main, dev]

permissions:
  contents: read
  actions: read

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

jobs:
  # -----------------------------------------------------------------------
  # Unit + security tests (fast, no network, no heavy optional deps)
  # -----------------------------------------------------------------------
  unit-tests:
    name: Unit & Security Tests (Python ${{ matrix.python-version }})
    runs-on: ubuntu-latest

    strategy:
      fail-fast: false
      matrix:
        python-version: ["3.10", "3.11", "3.12"]

    steps:
      - uses: actions/checkout@v7

      - name: Set up Python ${{ matrix.python-version }}
        uses: actions/setup-python@v6
        with:
          python-version: ${{ matrix.python-version }}
          cache: pip

      - name: Install core dependencies
        run: |
          python -m pip install --upgrade pip
          # Install package without optional heavy extras (no rag, no playwright)
          pip install -e ".[dev]"

      - name: Run unit tests with coverage
        env:
          # Provide dummy keys so settings initialise without errors
          OPENAI_API_KEY: sk-test-openai-placeholder
          ANTHROPIC_API_KEY: sk-ant-test-placeholder
          PENTESTAGENT_MODEL: gpt-4o
        run: |
          pytest tests/unit/ tests/security/ \
            --tb=short \
            -v \
            --cov=pentestagent \
            --cov-report=xml:coverage-unit.xml \
            --cov-report=term-missing \
            --cov-fail-under=30

      - name: Upload unit coverage report
        if: matrix.python-version == '3.12'
        uses: actions/upload-artifact@v7
        with:
          name: coverage-unit-py${{ matrix.python-version }}
          path: coverage-unit.xml
          retention-days: 7

  # -----------------------------------------------------------------------
  # Integration tests (require more setup, run separately)
  # -----------------------------------------------------------------------
  integration-tests:
    name: Integration Tests (Python 3.12)
    runs-on: ubuntu-latest
    needs: unit-tests

    steps:
      - uses: actions/checkout@v7

      - name: Set up Python 3.12
        uses: actions/setup-python@v6
        with:
          python-version: "3.12"
          cache: pip

      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install -e ".[dev]"

      - name: Run integration tests
        env:
          OPENAI_API_KEY: sk-test-openai-placeholder
          ANTHROPIC_API_KEY: sk-ant-test-placeholder
          PENTESTAGENT_MODEL: gpt-4o
        run: |
          pytest tests/integration/ \
            --tb=short \
            -v \
            --cov=pentestagent \
            --cov-report=xml:coverage-integration.xml

      - name: Upload integration coverage report
        uses: actions/upload-artifact@v7
        with:
          name: coverage-integration
          path: coverage-integration.xml
          retention-days: 7

  # -----------------------------------------------------------------------
  # Full coverage report (merged from both jobs) - only on PRs to main
  # -----------------------------------------------------------------------
  coverage-report:
    name: Combined Coverage Report
    runs-on: ubuntu-latest
    needs: [unit-tests, integration-tests]
    if: github.event_name == 'pull_request' && github.base_ref == 'main'

    steps:
      - uses: actions/checkout@v7

      - name: Set up Python 3.12
        uses: actions/setup-python@v6
        with:
          python-version: "3.12"
          cache: pip

      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install -e ".[dev]"

      - name: Download coverage artifacts
        uses: actions/download-artifact@v8
        with:
          pattern: coverage-*
          merge-multiple: true

      - name: Combine and report coverage
        env:
          OPENAI_API_KEY: sk-test-openai-placeholder
          ANTHROPIC_API_KEY: sk-ant-test-placeholder
          PENTESTAGENT_MODEL: gpt-4o
        run: |
          pytest tests/ \
            --tb=short \
            -q \
            --cov=pentestagent \
            --cov-report=xml:coverage-combined.xml \
            --cov-report=term-missing \
            --cov-fail-under=30

      - name: Upload combined coverage
        uses: actions/upload-artifact@v7
        with:
          name: coverage-combined
          path: coverage-combined.xml
          retention-days: 30

  # -----------------------------------------------------------------------
  # Lint (runs in parallel with tests, does not block them)
  # -----------------------------------------------------------------------
  lint:
    name: Lint (ruff + black check)
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v7

      - name: Set up Python 3.12
        uses: actions/setup-python@v6
        with:
          python-version: "3.12"
          cache: pip

      - name: Install linting tools
        run: |
          python -m pip install --upgrade pip
          pip install ruff black

      - name: Run ruff
        run: ruff check pentestagent/ --output-format=github
        continue-on-error: true

      - name: Check formatting with black
        run: black --check pentestagent/
        continue-on-error: true

The same workflow, on Latchkey

Removes redundant runs and caps runaway jobs. Added and changed lines are highlighted.

name: Tests
 
on:
  push:
    branches: ["**"]
  pull_request:
    branches: [main, dev]
 
permissions:
  contents: read
  actions: read
 
concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true
 
jobs:
  # -----------------------------------------------------------------------
  # Unit + security tests (fast, no network, no heavy optional deps)
  # -----------------------------------------------------------------------
  unit-tests:
    timeout-minutes: 30
    name: Unit & Security Tests (Python ${{ matrix.python-version }})
    runs-on: latchkey-small
 
    strategy:
      fail-fast: false
      matrix:
        python-version: ["3.10", "3.11", "3.12"]
 
    steps:
      - uses: actions/checkout@v7
 
      - name: Set up Python ${{ matrix.python-version }}
        uses: actions/setup-python@v6
        with:
          python-version: ${{ matrix.python-version }}
          cache: pip
 
      - name: Install core dependencies
        run: |
          python -m pip install --upgrade pip
          # Install package without optional heavy extras (no rag, no playwright)
          pip install -e ".[dev]"
 
      - name: Run unit tests with coverage
        env:
          # Provide dummy keys so settings initialise without errors
          OPENAI_API_KEY: sk-test-openai-placeholder
          ANTHROPIC_API_KEY: sk-ant-test-placeholder
          PENTESTAGENT_MODEL: gpt-4o
        run: |
          pytest tests/unit/ tests/security/ \
            --tb=short \
            -v \
            --cov=pentestagent \
            --cov-report=xml:coverage-unit.xml \
            --cov-report=term-missing \
            --cov-fail-under=30
 
      - name: Upload unit coverage report
        if: matrix.python-version == '3.12'
        uses: actions/upload-artifact@v7
        with:
          name: coverage-unit-py${{ matrix.python-version }}
          path: coverage-unit.xml
          retention-days: 7
 
  # -----------------------------------------------------------------------
  # Integration tests (require more setup, run separately)
  # -----------------------------------------------------------------------
  integration-tests:
    timeout-minutes: 30
    name: Integration Tests (Python 3.12)
    runs-on: latchkey-small
    needs: unit-tests
 
    steps:
      - uses: actions/checkout@v7
 
      - name: Set up Python 3.12
        uses: actions/setup-python@v6
        with:
          python-version: "3.12"
          cache: pip
 
      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install -e ".[dev]"
 
      - name: Run integration tests
        env:
          OPENAI_API_KEY: sk-test-openai-placeholder
          ANTHROPIC_API_KEY: sk-ant-test-placeholder
          PENTESTAGENT_MODEL: gpt-4o
        run: |
          pytest tests/integration/ \
            --tb=short \
            -v \
            --cov=pentestagent \
            --cov-report=xml:coverage-integration.xml
 
      - name: Upload integration coverage report
        uses: actions/upload-artifact@v7
        with:
          name: coverage-integration
          path: coverage-integration.xml
          retention-days: 7
 
  # -----------------------------------------------------------------------
  # Full coverage report (merged from both jobs) - only on PRs to main
  # -----------------------------------------------------------------------
  coverage-report:
    timeout-minutes: 30
    name: Combined Coverage Report
    runs-on: latchkey-small
    needs: [unit-tests, integration-tests]
    if: github.event_name == 'pull_request' && github.base_ref == 'main'
 
    steps:
      - uses: actions/checkout@v7
 
      - name: Set up Python 3.12
        uses: actions/setup-python@v6
        with:
          python-version: "3.12"
          cache: pip
 
      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install -e ".[dev]"
 
      - name: Download coverage artifacts
        uses: actions/download-artifact@v8
        with:
          pattern: coverage-*
          merge-multiple: true
 
      - name: Combine and report coverage
        env:
          OPENAI_API_KEY: sk-test-openai-placeholder
          ANTHROPIC_API_KEY: sk-ant-test-placeholder
          PENTESTAGENT_MODEL: gpt-4o
        run: |
          pytest tests/ \
            --tb=short \
            -q \
            --cov=pentestagent \
            --cov-report=xml:coverage-combined.xml \
            --cov-report=term-missing \
            --cov-fail-under=30
 
      - name: Upload combined coverage
        uses: actions/upload-artifact@v7
        with:
          name: coverage-combined
          path: coverage-combined.xml
          retention-days: 30
 
  # -----------------------------------------------------------------------
  # Lint (runs in parallel with tests, does not block them)
  # -----------------------------------------------------------------------
  lint:
    timeout-minutes: 30
    name: Lint (ruff + black check)
    runs-on: latchkey-small
 
    steps:
      - uses: actions/checkout@v7
 
      - name: Set up Python 3.12
        uses: actions/setup-python@v6
        with:
          python-version: "3.12"
          cache: pip
 
      - name: Install linting tools
        run: |
          python -m pip install --upgrade pip
          pip install ruff black
 
      - name: Run ruff
        run: ruff check pentestagent/ --output-format=github
        continue-on-error: true
 
      - name: Check formatting with black
        run: black --check pentestagent/
        continue-on-error: true

What changed

Run on Latchkey managed runners with one line (runs-on), which apply the fixes below automatically and self-heal transient failures. This example uses latchkey-small; pick the runner size that fits the job.
Add a job timeout so a hung step cannot burn hours of runner time.

What Latchkey heals here

This workflow has steps that commonly fail on transient issues (network, registries, flaky browsers). On Latchkey managed runners they are detected, retried, and self-healed instead of failing your build:

Dependency installs
End-to-end and browser tests

This workflow runs 4 jobs (6 with the matrix expanded) per trigger. On Latchkey the same minutes cost up to 58% less than GitHub-hosted, with zero queue time.

Actions used in this workflow

actions/checkout actions/setup-python actions/upload-artifact actions/download-artifact