What Is a Flaky Test?
A flaky test is one that produces inconsistent results, sometimes passing and sometimes failing, without any change to the code under test. Common causes include race conditions, reliance on timing, shared state between tests, or unstable external dependencies. Flaky tests erode trust in CI because a red build no longer reliably means a real problem.
Why it matters
Flaky tests train teams to ignore failures and re-run until green, which masks genuine bugs and wastes runner minutes. Clean, isolated, ephemeral environments remove an entire category of flakiness caused by leftover state, while quarantining and fixing flaky tests restores confidence in the suite.
Related concepts
- Test isolation removes shared-state flakiness
- Ephemeral runners prevent cross-job contamination
- Retries can mask flakiness instead of fixing it
Related guides
What Is a Regression Test?A regression test verifies that previously working functionality still works after a change, catching newly i…
What Is an Ephemeral Runner?An ephemeral runner is a CI worker that runs exactly one job on a fresh machine, then is destroyed, giving ev…
What Is Retry with Backoff?Retry with backoff re-attempts a failed operation after progressively longer waits, often with jitter, to han…