CI "No space left on device" with Free Bytes (Inode Exhaustion)

A confusing ENOSPC: df -h shows free space, yet writes fail with "No space left on device". The filesystem ran out of inodes (the metadata slots for files), not bytes. Usually your own output created millions of tiny files.

What this error means

Creating a file fails with No space left on device, but df -h reports free space. df -i shows IUse% at 100% and IFree at 0, from huge dependency trees, caches, or generated files.

$ touch x
touch: cannot touch 'x': No space left on device
$ df -i /
Filesystem  Inodes  IUsed IFree IUse% Mounted on
/dev/root  2560000 2560000     0  100% /

Common causes

Too many small files exhausted the inode table

Each file uses one inode regardless of size. Huge dependency trees, package caches, or generated files use up every inode while leaving most bytes free.

A fixed inode count set at filesystem creation

ext4 inode counts are fixed at mkfs time, so a low inode ratio runs out of inodes long before bytes.

How to fix it

Confirm it is inodes, then delete file-heavy dirs

Check inode usage, find the directories holding the most files, and remove them. This is a deterministic fix in your own job output.

df -i
for d in /*; do echo "$(find "$d" -xdev 2>/dev/null | wc -l) $d"; done | sort -rn | head
rm -rf node_modules ~/.cache/pip ~/.npm /tmp/*

Generate fewer tiny files

1. Avoid unpacking caches into the work volume when not needed.
2. Bundle or archive large numbers of small generated files.
3. A larger runner with a higher inode count buys headroom, but the root cause is the file count your job produces.

How to prevent it

• Watch df -i, not just df -h, in CI.
• Keep generated file counts bounded.
• Provision filesystems with adequate inode ratios.

Related guides