How to Profile a Slow Test Suite in CI

A slow suite is usually a handful of heavy tests, not uniform slowness. Profiling per-test timing shows exactly where to spend effort.

Before parallelizing or caching, profile the suite. Most slow suites have a long tail: a few specs dominate, or setup and teardown cost more than the tests. Per-test timing tells you which.

1. Get per-test timing

Most runners can report the slowest tests. Start there before any optimization.

- run: npx jest --silent --reporters=default --testTimeout=30000 \
    && npx jest --listTests
- run: pytest --durations=20

2. Separate test time from setup time

A suite can be slow because each test rebuilds a fixture, spins up a container, or migrates a database. Measure setup/teardown separately; shared, once-per-suite setup often beats per-test setup.

3. Attack the long tail first

If 5 specs are 60 percent of the time, splitting or fixing those beats micro-optimizing the fast ones. Use the timing report to target the heaviest specs.

4. Profile on a stable runner

Timing is meaningless on a noisy, throttled runner where CPU varies run to run. A right-sized Latchkey runner gives consistent CPU so your profile reflects the code, not noisy-neighbor jitter.

Key takeaways

• Profile per-test timing before parallelizing or caching.
• Separate test time from setup/teardown; shared setup often wins.
• Attack the long tail of a few heavy specs first.

Related guides