Python "RecursionError: maximum recursion depth exceeded" in CI

Python capped the call stack to protect against a C-stack overflow and raised RecursionError. Either the recursion is genuinely unbounded, or a deep-but-finite structure (pickling, deepcopy, recursive descent) exceeded the default limit.

What this error means

A run aborts with RecursionError: maximum recursion depth exceeded, often inside pickle, copy.deepcopy, a recursive serializer, or your own recursive function. It may appear only in CI when the data is larger than local fixtures.

RecursionError: maximum recursion depth exceeded while calling a Python object
  File "serialize.py", line 22, in encode
    return [encode(v) for v in obj]

Common causes

Unbounded or missing base case

A recursive function lacks a terminating condition (or a cyclic structure has no cycle guard), so it recurses until the limit is hit.

Deep but finite structure over the default limit

Pickling, deepcopy, or recursive descent over a deeply nested object can exceed CPython’s default limit (~1000) even though the recursion is correct.

How to fix it

Fix the recursion or add a cycle guard

For unbounded recursion, add a base case; for cyclic data, track visited nodes. This is a logic bug, not a transient failure.

def encode(obj, _seen=None):
    _seen = _seen or set()
    if id(obj) in _seen:        # break cycles
        raise ValueError("cycle detected")
    _seen.add(id(obj))
    ...

Raise the limit for genuinely deep data

When the recursion is correct but deep, raise the limit deliberately (and mind the C-stack).

import sys
sys.setrecursionlimit(10_000)   # for known-deep, finite structures

Rewrite hot recursion iteratively

1. Convert deep tree/list traversal to an explicit stack/queue loop.
2. Use generators or itertools to flatten without growing the call stack.
3. Reserve setrecursionlimit for cases an iterative form is impractical.

How to prevent it

• Always give recursive functions a base case and cycle protection.
• Prefer iterative traversal for arbitrarily deep structures.
• Test with CI-sized (not just local) data to surface depth issues.

Related guides