Split slow test suite CI jobs without losing the test report

By Kaveh Alemi·Latchkey

You can split slow test suite CI jobs in about four lines of YAML, and the four lines are the easy half. The half that takes an afternoon is putting the results back: four uploads that do not collide, one report instead of four, a coverage total that is actually a total, and a single check the branch rule can point at.

Four shards uploading uniquely named artifacts into one merge job that emits report, coverage and verdict — What a split suite has to reassemble. Action names and inputs read from upload-artifact v7.0.1 and download-artifact v8.0.1 on 2026-09-21.

A suite that takes twenty minutes blocks every merge for twenty minutes, and the fix is mechanical: run quarters of it on four machines. Every test runner in common use ships a flag for that, and the split itself is rarely where people get stuck.

What breaks is everything downstream. The test report is now four reports, the coverage number is four partial numbers, one artifact name is being uploaded four times, and the required status check names a job that no longer exists as a single job. This page is that second half.

The split, briefly

Almost every runner takes the same shape, a one-based index over a total, so a matrix leg maps onto it directly: --shard=N/4 for vitest, jest and Playwright, --splits 4 --group N for pytest with a splitting plugin. Take the total from the strategy.job-total context rather than writing the number twice, so widening the matrix cannot leave the two out of step.

Two settings are not optional. fail-fast: false keeps the other shards running when one fails, which is the difference between learning that one test broke and learning that the suite is broken. And each shard needs its own artifact name, for a reason the next section is entirely about. How many shards to use, and how to keep them balanced, are covered in test sharding across runners and parallelizing by timing.

Every shard needs its own artifact name

This is the first thing that breaks and the error does not say so clearly. The upload action's own README states the rule: artifact names must be unique since each created artifact is idempotent so multiple jobs cannot modify the same artifact. It adds the matrix case explicitly, warning that uploading to the same artifact will produce conflict errors and advising a prefix or suffix from the matrix.

The message you get back is assembled at runtime and exists as a literal in no source file, which is why searching for the whole line finds nothing. The client in @actions/artifact builds it from a fixed prefix, the HTTP status, the status message and a msg field returned by GitHub's artifact service, which is closed. Recognize the first fragment, then read the tail for the reason.

Reconstructed from the two format strings in actions/toolkit packages/artifact/src/internal/shared/artifact-twirp-client.ts; the explanatory text that follows this prefix is returned by GitHub's artifact service and is in no client source

Received non-retryable error: Failed request: (409) Conflict

Collect them in one job

The merge job needs every shard whether it passed or not, which means needs on the matrix job and if: always(), since a failed shard would otherwise skip the very job that explains what failed. Downloading without a name gets all of the artifacts from the run, one directory each.

One input decides how much shell you write afterwards. Left alone, each artifact is extracted into its own named directory; merge-multiple: true puts them all in the same directory instead. For four files called junit-1.xml through junit-4.xml the flat layout is what you want, and it is the difference between a glob and a find.

.github/workflows/ci.yml

  report:
    needs: test
    if: always()
    runs-on: latchkey-small
    steps:
      - uses: actions/download-artifact@v8
        with:
          pattern: junit-*
          path: reports
          merge-multiple: true
      - run: ls -R reports && npx junit-merge -d reports -o junit.xml

One report, one coverage number, per tool

Merging is per ecosystem and the tools are not interchangeable, so the table below is the shortest honest version. The important split is between merging test results, which is mostly an XML concatenation, and merging coverage, which is not: coverage files hold per-line counters that have to be summed, and concatenating them gives a number that is wrong in a direction that flatters you.

Playwright is the one worth copying. Its blob reporter records everything that ran along with traces and attachments, and merge-reports turns a directory of blobs into one ordinary HTML report, so the merged artifact is the same thing a single run would have produced. Where your runner has no equivalent, emit JUnit XML for the results and a native coverage file for the counters, and merge the two separately.

Runner	Per-shard output	Merge the results with	Merge the coverage with
Playwright	Blob report directory	`playwright merge-reports`	Its own coverage tooling, separately
pytest	`--junitxml` plus `.coverage`	A JUnit merge step	`coverage combine` then `coverage report`
Jest or Vitest	JUnit XML plus `coverage-final.json`	A JUnit merge step	`nyc merge` then a report step
Go	`go test -json` output	`go tool test2json` consumers	`go tool covdata merge`

The required check has to be the aggregate

Before the split, the branch rule named one job. After it, that name belongs to four jobs with matrix suffixes, and the rule matches none of them. The usual fix is a small job that needs the matrix and reports the verdict, which the branch rule names instead.

Write the condition carefully, because the default is wrong in a way that passes. A job whose needs failed is itself skipped, and a skipped job reports "Success" to a required check, so an aggregate written without if: always() will go green precisely when the suite went red. Check the results explicitly rather than trusting the job to have been reached at all.

.github/workflows/ci.yml

  verdict:
    needs: test
    if: always()
    runs-on: ubuntu-latest
    steps:
      - name: Fail unless every shard succeeded
        run: |
          echo "shards reported: ${{ needs.test.result }}"
          test "${{ needs.test.result }}" = "success"

Where the split stops being worth it

Each shard repeats the whole setup, so the question is whether your work is large compared with that setup. It is easy to underestimate. On a runner we measured, running a single one-second test file took 1.887 seconds end to end, which puts the test runner's own startup at about 0.9 seconds, and a dependency install cost 1.714 seconds cold against 0.854 with a warm cache. That is roughly a second and three quarters per shard warm, and two and a half cold, before a single test of yours runs.

Then the bill rounds. Each job is billed up to the whole minute, so four shards that each finish in forty seconds are four billed minutes against one, no matter how good the split is. Short suites lose that trade and long ones barely notice it, which is the arithmetic rather than an opinion.

Why no recorded run backs this page

The timings in the section above are quoted from job-n.log, committed under content/repro/timings/shard-tests-across-github-actions-runners/ from a run on a Latchkey latchkey-small runner on 20 September 2026, with its script and a status file naming the job id and exit code. No new runner job was run for this page, and the merge machinery above was not exercised on one.

That is a real gap and worth naming precisely. What a harness would have to record here is a four-way matrix, a deliberate name collision, and a branch rule, and the last of those is repository configuration rather than anything a job can print. So the artifact behavior above is read out of the two actions' own READMEs and the error prefix out of the toolkit source, and the merge commands are named rather than timed.

Frequently asked questions

Why do my matrix jobs fail when uploading the same artifact name?

Because artifact names have to be unique within a run. The upload action documents that each created artifact is idempotent, so multiple jobs cannot modify the same one, and its README warns specifically about matrix scenarios, advising a prefix or suffix from the matrix in the name. The error comes back as a 409 conflict that the client treats as non-retryable, so it fails immediately.

How do I merge JUnit reports from parallel CI jobs?

Upload one uniquely named report per shard, then add a job that needs the matrix, runs with if: always() so a failed shard does not skip it, and downloads with merge-multiple: true so everything lands in one directory. Merge the XML there. Playwright is the exception worth using: its blob reporter plus merge-reports rebuilds the report a single run would have produced.

How do I combine code coverage across shards?

Not by concatenating the files. Coverage formats carry per-line counters that have to be summed, so use the ecosystem tool: coverage combine for Python, nyc merge for JavaScript, go tool covdata merge for Go. Merge the coverage separately from the test results, because the two use different files and different tools even when one job does both.

What do I point a required status check at after sharding?

A small aggregate job that needs the matrix, since the branch rule can no longer match the per-leg names. Give it if: always() and test needs.<job>.result explicitly, because a job whose dependency failed is skipped, and a skipped job reports "Success" to a required check. Without that condition the aggregate goes green exactly when the suite went red.