How to Debug a Flaky Test by Re-running the Step in GitHub Actions

Looping a suspect test dozens of times in one step forces an intermittent failure to show up while you watch.

Run the test in a shell loop (or with a runner repeat flag) so a flake that fails one time in twenty is exposed in a single run, instead of needing many separate re-runs.

Steps

• Pick the one suspect test, not the whole suite.
• Loop it many times so the flake has a chance to fire.
• Stop on the first failure and capture the output.

Workflow

steps:
  - name: Hammer one test to expose flakiness
    run: |
      for i in $(seq 1 50); do
        echo "::group::run $i"
        npx jest path/to/flaky.test.js --runInBand || exit 1
        echo "::endgroup::"
      done
  # pytest has a built-in repeat plugin:
  - run: pytest tests/test_flaky.py --count=50 -x

Gotchas

• exit 1 on the first failure stops the loop so the log ends on the failing iteration.
• pytest --count needs the pytest-repeat plugin installed.
• Latchkey can auto-retry transient flakes on managed runners, but reproducing the flake first tells you whether it is the test or the environment.

Related guides