Self-Healing CI: Recovering "fork: Cannot Allocate Memory"

A failed fork is the runner saying it has no room for another process - the fix is more headroom or fewer concurrent processes, not a change to your code.

The problem

A step fails with fork: Cannot allocate memory or Resource temporarily unavailable while spawning a process. The build is fine; the runner hit a memory ceiling or a process/PID limit. A human re-runs on a bigger machine or lowers parallelism and it passes unchanged.

Typical symptom

sh: fork: Cannot allocate memory
# or
java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached

Why it happens

Creating a process requires available memory and a free process slot. When a build forks many workers, or memory is already tight, the kernel refuses the next fork even though no single process is huge - the ceiling is on the aggregate, not on your code.

Per-user process/PID limits and overcommit settings cap how many processes a job can spawn, so a highly parallel step can exhaust slots and fail to fork while leaving plenty of CPU idle.

The manual fix

The manual fix is to lower process pressure or add resources, then retry:

Reduce build/test parallelism so fewer processes are spawned at once.
Re-run on a runner with more memory and a higher process/PID allowance.
Cap per-tool worker counts (e.g. job/thread limits) so the step stays under the ceiling.

How this gets automated

A failed fork has a clear resource-exhaustion signature, and the right response is well-defined: retry with adequate memory and process headroom rather than masking a bug. A self-healing CI pipeline recognizes the allocation/PID-limit condition, retries the step with the resources it needs, and only escalates if the failure is not resource-driven - so the developer never sees the red build.