How to Shard Tests Across CI Runners Without Wasting Minutes

By Kaveh Alemi·Latchkey

Sharding trades money for wall-clock time, and it stops paying at a point you can calculate. Past that point you are buying setup overhead, not speed.

A twenty-minute test suite across four shards does not take five minutes. Each shard repeats the fixed cost of the job: checkout, toolchain setup, dependency install, and browser or service startup. That fixed cost is paid N times while only the test execution is divided.

That is why sharding has a knee. Work out where yours is before adding shards, because past it you are paying for more jobs and getting no faster.

The arithmetic

Total wall-clock per shard is fixed overhead plus execution divided by shard count. Total billed minutes is that, multiplied by the shard count.

Shards	Wall clock (2 min setup, 20 min tests)	Billed minutes
1	22 min	22
2	12 min	24
4	7 min	28
8	4.5 min	36
16	3.25 min	52

Shard by timing, not by file count

Splitting evenly by file count gives uneven shards, and the job finishes when the slowest shard finishes. Most runners can split by recorded timing instead, which is the single biggest improvement available.

Terminal

# Vitest
npx vitest run --shard=${{ matrix.shard }}/${{ strategy.job-total }}

# Playwright (balances by test, not file)
npx playwright test --shard=${{ matrix.shard }}/4

# Jest, using timing data from a previous run
npx jest --shard=${{ matrix.shard }}/4

# pytest, split by recorded durations
pytest --splits 4 --group ${{ matrix.group }} --durations-path .test_durations

A shard matrix that does not waste setup

.github/workflows/ci.yml

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false          # one bad shard should not cancel the rest
      matrix:
        shard: [1, 2, 3, 4]
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 22, cache: npm }   # cache is per-shard; keep it warm
      - run: npm ci
      - run: npx vitest run --shard=${{ matrix.shard }}/4
      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: results-${{ matrix.shard }}
          path: junit-*.xml

Reduce the fixed cost before adding shards

Cache dependencies and key the cache to the lockfile hash. Install is usually the largest part of per-shard overhead.
Install browsers or services only in the shards that need them.
If setup is 2 minutes and tests are 3, sharding is the wrong tool; the setup is the problem.
Merge shard reports into one artifact so a failure is diagnosable without opening four job logs.

Measure before you optimise

Pipeline optimisation usually targets the step people assume is slow. Get the real per-step timings first, because the answer is frequently dependency install or a cold cache rather than the build itself.

Terminal

# per-job timings for the last 20 runs
gh run list --limit 20 --json databaseId,conclusion,createdAt,updatedAt \
  --jq '.[] | "\(.conclusion)\t\(.createdAt)\t\(.updatedAt)"'

# per-step timing inside one run
gh run view <run-id> --log | grep -E "^\S+\s+.*Run |##\[group\]" | head -40

Frequently asked questions

How many shards should I use?

Compute the knee: wall clock per shard is fixed setup plus execution divided by shards, while billed minutes multiply by shard count. Most suites stop benefiting between 4 and 8. If setup is a large share of the job, the useful number is lower.

Why is my sharded suite not proportionally faster?

Because each shard repeats the fixed cost of checkout, setup, and install. Only the execution portion divides. If setup is 2 minutes and tests are 3, four shards saves you barely two minutes and costs four times the setup.

Should I shard by file or by timing?

By timing wherever the runner supports it. File-count splits produce uneven shards and the job only finishes when the slowest one does, so one heavy file can erase most of the benefit.

Does sharding cost more money?

Yes. It trades billed minutes for wall-clock time, and per-minute rounding makes it worse because every short shard rounds up. That trade is usually worth it for merge latency, but it is a trade, not a saving.