# Self-Healing CI: Recovering from Inode / Disk-Pressure Exhaustion

> Running out of inodes fails writes even with free bytes left. See the manual fix and how self-healing CI reclaims inodes and retries automatically.

Source: https://latchkey.dev/learn/self-healing-ci/self-heal-inode-exhaustion  
Updated: 2026-06-25

A disk can be far from full on bytes yet still refuse writes because it ran out of inodes - the fix is to clear small files, not to add gigabytes.

## What makes a failure safely retryable

Automatic retry is only correct for failures that are genuinely transient. Retrying a deterministic failure wastes minutes and hides a real defect, so the classification matters more than the retry mechanism.

- Safe to retry: network timeouts, registry 5xx, transient DNS failures, a service container that was not ready, a spot instance reclaimed mid-run.
- Not safe to retry: assertion failures, compile errors, lint violations, anything that fails identically on every attempt.
- Ambiguous, and worth investigating rather than retrying: out-of-memory kills, disk exhaustion, and flaky tests. These repeat under load and a retry only hides the trend.
- Always record that a retry happened. A pipeline that silently retries is a pipeline whose real failure rate you do not know.

## FAQ

### Why does the disk say it has free space but writes still fail?

Because byte capacity and inode capacity are separate limits. `df -h` shows bytes; `df -i` shows inodes. A workload that creates a flood of tiny files can hit 100% inode usage with gigabytes of bytes still free, and the only fix is to delete files, not to add storage.

---

Latchkey runs CI/CD that repairs its own failures. Agent entry points: https://latchkey.dev/agent.txt, https://latchkey.dev/openapi.json, https://latchkey.dev/llms.txt