How to Quarantine Flaky Tests Without Hiding Real Bugs

Quarantine is the discipline of pulling a known-flaky test out of the merge-blocking path without deleting it - so the build stays trustworthy while you keep the test on the books to fix.

When a test is provably flaky, you have three bad options (ignore it, delete it, or let it block merges) and one good one: quarantine. Done right it removes the noise without losing the coverage.

Why quarantine instead of delete or retry-forever

Deleting a flaky test loses the coverage it provides when it is not flaking. Blanket-retrying hides genuine regressions. Letting it block merges trains the team to ignore red builds. Quarantine threads the needle: the test still runs and reports, but its result no longer gates the merge.

How to quarantine

Tag the test (e.g. a quarantine/flaky marker) so the runner separates it.
Run quarantined tests in a non-blocking lane that reports but does not fail the build.
File a tracking ticket linked to the test the moment you quarantine it.
Keep the test executing so you collect data on how often it still flakes.

Keep it honest

Quarantine must be temporary and visible, or it becomes a graveyard of dead coverage. Cap how long a test may stay quarantined, surface the quarantine list on a dashboard, and require a fix-or-delete decision when the cap expires. The principle from flake management holds: contain the noise, never silence the signal.

Promoting back or removing

When the root nondeterminism is fixed, return the test to the blocking suite and confirm it is stable across many runs. If it cannot be made deterministic and tests nothing unique, delete it deliberately - an honest removal beats a permanently quarantined ghost.

Key takeaways

Quarantine moves a known-flaky test out of the blocking path but keeps it running.
It preserves coverage without letting noise gate merges or hide regressions.
Always pair quarantine with a tracking ticket and a time cap.
Promote back when fixed, or delete deliberately - never leave a permanent ghost.