# Self-Healing CI: Recovering When the Runner Fills Up Mid-Build

> A runner that fills its disk partway through a build hit a capacity ceiling, not a code bug. See the manual cleanup and how self-healing CI reclaims space and retries.

Source: https://latchkey.dev/learn/self-healing-ci/self-healing-disk-full-mid-build  
Updated: 2026-06-26

When a build runs fine until the disk fills, the failure is about capacity, not your code -- reclaim space and the same commit completes.

## What makes a failure safely retryable

Automatic retry is only correct for failures that are genuinely transient. Retrying a deterministic failure wastes minutes and hides a real defect, so the classification matters more than the retry mechanism.

- Safe to retry: network timeouts, registry 5xx, transient DNS failures, a service container that was not ready, a spot instance reclaimed mid-run.
- Not safe to retry: assertion failures, compile errors, lint violations, anything that fails identically on every attempt.
- Ambiguous, and worth investigating rather than retrying: out-of-memory kills, disk exhaustion, and flaky tests. These repeat under load and a retry only hides the trend.
- Always record that a retry happened. A pipeline that silently retries is a pipeline whose real failure rate you do not know.

## FAQ

### What causes Self-Healing CI: recovering when the runner fills up Mid-Build?

A build progresses normally and then fails because the runner ran out of disk space partway through -- intermediate artifacts, layers, and caches accumulated until a write failed. A human adds a cleanup step or re-runs on a fresh runner and the build passes unchanged.

### How do I fix Self-Healing CI: recovering when the runner fills up Mid-Build manually?

[object Object]

### Can Self-Healing CI: recovering when the runner fills up Mid-Build be fixed automatically?

A mid-build disk-exhaustion failure has a clear signature and a well-defined remedy: reclaim space, then retry. A self-healing CI pipeline detects the out-of-space condition, frees space on the runner with safe cleanup or retries with adequate capacity, and can suggest a durable change so the build stops outgrowing its disk.

---

Latchkey runs CI/CD that repairs its own failures. Agent entry points: https://latchkey.dev/agent.txt, https://latchkey.dev/openapi.json, https://latchkey.dev/llms.txt