Playwright Flaky Test - Passes on Retry in CI

A Playwright test fails on its first attempt and passes on a configured retry, so Playwright marks it "flaky." Usually a race: asserting before the UI settles, a fixed sleep, or shared state between tests.

What this error means

The HTML report shows the test as "flaky" - failed then passed - and CI is green overall. The failure moves around between runs, the signature of a timing race rather than a real bug.

playwright

Running 120 tests using 4 workers

  ✓  tests/cart.spec.ts:12:1 › adds to cart (retry #1)
  1 flaky
    tests/cart.spec.ts:12:1 › adds to cart

Common causes

Asserting before the app settles

A non-retrying check runs right after an action that triggers async work (XHR, animation), so it sometimes reads a pre-update state.

Fixed sleeps and shared state

A waitForTimeout(500) is sometimes too short, or tests share storage/server state, making outcomes order- and timing-dependent.

How to fix it

Use web-first, auto-retrying assertions

Assert on the readiness signal so the test waits exactly as long as needed instead of a fixed sleep.

cart.spec.ts

await page.getByRole('button', { name: 'Add' }).click();
await expect(page.getByTestId('cart-count')).toHaveText('1');

Isolate state and capture a trace on retry

playwright.config.ts

// playwright.config.ts
export default defineConfig({
  retries: process.env.CI ? 2 : 0,
  use: { trace: 'on-first-retry' },
});

How to prevent it

Replace fixed sleeps with web-first expect assertions.
Give each test its own storage/server state.
Capture traces on first retry to debug intermittency fast.