Playwright flaky test "passed on retry #1" in CI

When retries is set, Playwright reruns a failed test and, if it passes, reports it as "flaky" rather than failed. The job stays green, but the reporter summary shows a flaky count that points at non-deterministic tests you should fix.

What this error means

The run ends green but the summary reads "N flaky" and individual tests show "Retry #1" that passed after a first attempt failed, usually on assertions or navigation timing.

Running 120 tests using 4 workers
  118 passed
  2 flaky
    [chromium] > checkout.spec.ts:24:5 > completes order (retry #1)

Common causes

A timing race that resolves on the second attempt

The first run loses a race (slow render, late network); the retry happens to win it, so the test is marked flaky rather than fixed.

Shared state leaking between tests

Tests that depend on order or shared data fail intermittently when parallel workers interleave, then pass when rerun in isolation.

How to fix it

Surface and investigate the flaky list

1. Read the reporter's flaky section to find which specs retried.
2. Open the retry's trace to see what differed between attempts.
3. Fix the underlying race (await, web-first assertion) rather than relying on the retry.

npx playwright test --trace on-first-retry

Isolate per-test state

Give each test its own data and context so parallel workers cannot interfere, removing order-dependent flake.

test.beforeEach(async ({ page }) => {
  await seedFreshUser();
});

How to prevent it

• Treat any flaky count as a bug to fix, not noise to ignore.
• Keep trace: on-first-retry so the flaky attempt is debuggable.
• Isolate per-test data to remove cross-test interference.

Related guides