Skip to content
Latchkey

Python "hypothesis Flaky / DeadlineExceeded" in CI

Hypothesis raises Flaky when an example fails then passes on replay (nondeterminism), and DeadlineExceeded when a single example takes longer than the default 200ms deadline, which slower CI runners hit easily.

What this error means

A property-based test fails with "hypothesis.errors.DeadlineExceeded: Test took N ms, which exceeds the deadline of 200.00 ms" or "hypothesis.errors.Flaky: Hypothesis ... produced unreliable results".

pytest
hypothesis.errors.DeadlineExceeded: Test took 412.55ms, which exceeds the deadline of 200.00ms

Common causes

The per-example deadline is too tight for CI

A slower CI runner exceeds the default 200ms deadline even though the logic is correct.

Hidden nondeterminism in the test

The test depends on time, randomness, or external state, so the same example does not behave consistently.

How to fix it

Relax or disable the deadline and remove nondeterminism

  1. Raise or disable the deadline with a settings profile for CI.
  2. Eliminate external state and clock/random dependence so examples are deterministic.
  3. Use @settings(deadline=...) on slow-but-correct tests.
Python
from hypothesis import settings

@settings(deadline=None)
def test_property(...):
    ...

How to prevent it

  • Set a CI hypothesis profile with a relaxed deadline.
  • Keep property tests free of time/random/external dependencies.
  • Pin and register a CI settings profile in conftest.

Related guides

Tired of flaky CI? Latchkey auto-heals failed jobs and retries them for you. Start free →