# How to run tests in parallel for coding agents

> Run tests in parallel for coding agents with a matrix, a dispatch per sub-agent or detached jobs, then wait on all of them and get one verdict back.

Source: https://latchkey.dev/learn/agent-ci/run-agent-tests-in-parallel  
Updated: 2026-11-12

To run tests in parallel for coding agents, fan the work out into separate jobs, then wait on all of them and reduce the results to one verdict the agent can branch on. On GitHub Actions that is a matrix or one `workflow_dispatch` run per sub-agent, watched one by one; with the Latchkey CLI it is `latchkey run --detach` under one batch and a single `latchkey wait`.

An agent that splits a task across sub-agents soon wants several test runs at the same moment: one per shard of a slow suite, one per candidate fix, one per worktree. Starting them is easy. Knowing when all of them are done, which ones failed, and keeping them from canceling each other or swallowing the team's concurrency is the actual work.

Below, the fan-out and the wait on GitHub Actions come first, with the limits that bite, and then the same shape with detached Latchkey jobs.

## Fan-out inside one run: the matrix

If every parallel piece runs the same workflow with a different parameter, a matrix is the cheapest fan-out GitHub has: one dispatch, one run id to watch, and GitHub expands the legs. A matrix generates at most 256 jobs per workflow run, and `max-parallel` caps how many legs run at once.

The default that hurts an agent is `fail-fast`. It defaults to `true`, and when it is on GitHub will cancel all in-progress and queued jobs in the matrix if any job in the matrix fails. A human wants that. An agent sharding a suite usually wants every shard's verdict, because the second failing shard is the one it would otherwise discover on the next attempt. Turn it off.

```.github/workflows/agent-shards.yml
on:
  workflow_dispatch:
jobs:
  test:
    runs-on: ubuntu-latest
    timeout-minutes: 20
    strategy:
      fail-fast: false
      max-parallel: 4
      matrix:
        shard: [1, 2, 3, 4]
    steps:
      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
      - run: npm ci
      - run: npm test -- --shard=${{ matrix.shard }}/4
```

> Matrix ceiling, `fail-fast` and `max-parallel` from the workflow syntax reference on docs.github.com, read 2026-11-12. The checkout SHA is the v7.0.1 tag from `git ls-remote https://github.com/actions/checkout`.

## Fan-out across runs: one dispatch per sub-agent

Sub-agents rarely share a parameter list. Each has its own branch or its own task, so each dispatches its own run. The REST endpoint behind `workflow_dispatch` now answers 200 with `workflow_run_id`, `run_url` and `html_url`, so the old dance of dispatching and then guessing the run id from `gh run list` is gone. Inputs are capped at 25 properties.

The join is where this gets slow. `gh run watch` takes one run id, so an agent with four sub-agents watches four runs in sequence. That is fine, because a run that already finished returns at once, and with `--exit-status` a finished run whose conclusion is not success exits non-zero. Collect every id first, then watch them all.

```Terminal
ids=()
for task in auth billing search export; do
  ids+=("$(gh api -X POST repos/{owner}/{repo}/actions/workflows/agent-check.yml/dispatches \
    -f ref="agent/$task" -f "inputs[task]=$task" --jq .workflow_run_id)")
done
failed=0
for id in "${ids[@]}"; do
  gh run watch "$id" --exit-status --compact >/dev/null || failed=1
done
exit $failed
```

> Response fields and the 25-input cap from the "Create a workflow dispatch event" REST reference; the already-completed behavior from `pkg/cmd/run/watch/watch.go` in cli/cli; `key[subkey]=value` field syntax from `gh api --help` in gh 2.102.0. All read 2026-11-12.

## Concurrency groups: how parallel runs cancel each other

This is the failure that looks like flakiness. Copy the usual `concurrency` block into an agent workflow, keyed on `github.ref`, and two sub-agents dispatching on the same branch now share a group. GitHub allows at most one running and one pending run per group, and by default a new pending run cancels the pending one before it. With `cancel-in-progress: true` the running one goes too. The agent reads a canceled run as a failed check and starts again.

Key the group on what makes the work distinct. The workflow-level `concurrency` expression may use the `github`, `inputs` and `vars` contexts, so a task input works. If the runs genuinely must take turns, `queue: max` lets up to 100 runs wait in the group instead of replacing each other; GitHub rejects it combined with `cancel-in-progress: true`.

Above all of this sits the plan's ceiling: on standard GitHub-hosted runners, 20 concurrent jobs on Free, 40 on Pro, 60 on Team and 500 on Enterprise, shared with every human push in the account. [Give your coding agent a CI runner](/learn/agent-ci/give-your-coding-agent-a-ci-runner) covers that ceiling and the macOS sub-limit in full.

```.github/workflows/agent-check.yml
on:
  workflow_dispatch:
    inputs:
      task:
        required: true
concurrency:
  group: agent-${{ github.workflow }}-${{ inputs.task }}
  cancel-in-progress: true
```

> `queue`, its 100-run cap and the allowed contexts from the workflow syntax reference; concurrent job limits from docs.github.com/en/actions/reference/limits. Both read 2026-11-12.

## The Latchkey way: detached jobs in one batch

With the Latchkey CLI there is no workflow to dispatch. `latchkey run --detach` packs the directory it runs from, submits the job to a fresh runner and prints the job id without waiting. A sub-agent in its own worktree therefore ships its own uncommitted edit, and `--context <dir>` packs another directory instead.

A batch is a name you put on every job of one piece of work, with `--batch` or the `LATCHKEY_BATCH` environment variable, which sub-agents inherit from the shell that starts them. Give the batch a budget and a limit on jobs running at once before the burst: jobs past that limit, or past the workspace's own limit, wait in the queue instead of being refused. `latchkey wait --batch` then waits on every job the batch holds when the wait starts, prints each outcome as it ends, then a table of state, exit code, run time, cost and batch, and exits 0 only when every job's exit code is 0.

```Terminal
export LATCHKEY_BATCH="refactor-auth-$(date +%Y%m%d%H%M%S)"
latchkey budget set "$LATCHKEY_BATCH" 3 --max-concurrent 4
for shard in 1 2 3 4; do
  latchkey run --detach "npm ci && npm test -- --shard=$shard/4"
done
latchkey wait --batch "$LATCHKEY_BATCH"
verdict=$?   # 0 only if every job in the batch exited 0
```

> Flags from `latchkey help run`, `latchkey help budget` and `latchkey help wait` in @latchkeydev/cli 0.16.0, run 2026-11-12.

## Reading the result without losing it

Two details decide whether the join is trustworthy. First, `--batch` on `wait` takes the jobs the batch holds when the wait starts, and it never reads `LATCHKEY_BATCH`, so start every job before you wait and pass the name explicitly. Second, read the verdict from the right field. With `--output json` the wait prints one `ended` event per job and then a `summary`; each carries `job_id`, `state`, `exit_code`, `run_seconds`, `cost_usd`, `batch` and `failure_reason`, and here `exit_code` is the verdict `latchkey run` would have given the job.

If the parallel pieces are Latchkey CI runs rather than one-off jobs, the same command waits on them by commit or pull request: `latchkey wait --pr <number> --repository <owner/name>`.

```Terminal
latchkey wait --batch "$LATCHKEY_BATCH" --output json >wait.ndjson
verdict=$?
jq -r 'select(.event == "ended") | "\(.job_id) \(.state) \(.exit_code) \(.failure_reason // "")"' <wait.ndjson
```

## The two fan-outs side by side

The limits differ more than the commands do.

|  | GitHub Actions | Latchkey CLI |
| --- | --- | --- |
| Unit of fan-out | A matrix leg, or one dispatched run | One detached job |
| Join | `gh run watch` once per run id | `latchkey wait --batch`, one command |
| What cancels what | Concurrency groups and `fail-fast` | Nothing by default; `latchkey cancel` per job |
| Ceiling on jobs at once | Plan limit: 20 Free, 40 Pro, 60 Team, 500 Enterprise | Workspace limit: 20 Developer, 40 Launch, 80 Scale; batch `--max-concurrent` |
| Over the ceiling | Jobs queue | Jobs wait in the queue with a `queue_reason` |
| Spend bound for the burst | None per burst | `latchkey budget set` on the batch |

> Latchkey workspace limits from latchkey.dev/documentation/runner-limits, read 2026-11-12.

## When not to fan out

Every parallel job pays its own dependency install and is billed in its own whole minutes, so a two-minute suite split four ways can cost more and finish no sooner. The workspace limit is shared by every agent and every workflow run, and Latchkey limits job creation per workspace per clock hour (120 on Developer, 240 on Launch, 600 on Scale). For a short suite, one job with several checks chained, `latchkey run 'npm ci && npm run lint && npm test'`, is the better shape. For the spend side of a burst, see [cap agent CI spend](/learn/agent-ci/cap-agent-ci-spend).

## Where the numbers come from

Every limit and flag here is quoted from a reference, not measured: docs.github.com for the matrix, concurrency and plan limits, the gh source and help for `gh run watch`, and `latchkey help` from CLI 0.16.0 with Latchkey's limits page for the batch side. No recorded run backs the page, and neither evidence file in this repository lists it. A timing captured from one burst would reflect that day's queue more than the shape of the fan-out, so none is offered.

## FAQ

### How do I wait for multiple workflow runs to complete in GitHub Actions?

Collect the run ids when you dispatch (the dispatch endpoint returns `workflow_run_id`), then run `gh run watch <id> --exit-status` for each one. A run that already finished returns immediately, so watching in sequence costs no extra time. Inside a single workflow, express the dependency with `needs:` instead.

### Why do my GitHub Actions runs get canceled when I start a new one?

They share a concurrency group. GitHub keeps at most one running and one pending run per group, a new pending run cancels the previous pending one, and `cancel-in-progress: true` also cancels the running one. Give each parallel run its own group, or set `queue: max` if they must take turns.

### How do I limit the number of parallel jobs in a GitHub Actions matrix?

Set `strategy.max-parallel` to the most legs you want running at once; the rest wait until a slot frees. It limits one matrix only, so two runs of the same workflow can still overlap unless a `concurrency` group keeps them apart.

### Does fail-fast cancel other matrix jobs?

Yes, by default. `strategy.fail-fast` defaults to `true`, and then GitHub cancels all in-progress and queued jobs in the matrix when any one fails. Set `fail-fast: false` when you want every leg's result.

## References

- [Workflow syntax: strategy and concurrency, docs.github.com (read 2026-11-12)](https://docs.github.com/en/actions/reference/workflows-and-actions/workflow-syntax)
- [REST API: create a workflow dispatch event](https://docs.github.com/en/rest/actions/workflows#create-a-workflow-dispatch-event)
- [GitHub Actions limits](https://docs.github.com/en/actions/reference/limits)
- [Latchkey CLI documentation (read 2026-11-12)](https://latchkey.dev/documentation/latchkey-cli)
- [Latchkey limits and concurrency](https://latchkey.dev/documentation/runner-limits)

---

Latchkey runs CI/CD that repairs its own failures. Agent entry points: https://latchkey.dev/agent.txt, https://latchkey.dev/openapi.json, https://latchkey.dev/llms.txt