Skip to content
Latchkey

CI "No space left on device" with Free Bytes - Inode Exhaustion

A confusing ENOSPC: df -h shows plenty of free space, yet writes still fail with "No space left on device". The filesystem ran out of *inodes* - the metadata slots for files - not out of bytes.

What this error means

Creating a file fails with No space left on device, but df -h reports free space. df -i tells the real story: IUse% is 100% and IFree is 0. Usually caused by millions of tiny files (node_modules, caches, package metadata).

Terminal
$ df -h /
Filesystem  Size  Used Avail Use% Mounted on
/dev/root    40G   18G   22G  46% /
$ touch x
touch: cannot touch 'x': No space left on device
$ df -i /
Filesystem  Inodes  IUsed IFree IUse% Mounted on
/dev/root  2560000 2560000     0  100% /

Common causes

Too many small files exhausted the inode table

Each file consumes one inode regardless of size. Huge dependency trees, package caches, or generated files can use up every inode while leaving most bytes free.

A fixed inode count set at filesystem creation

ext4 inode counts are fixed at mkfs time. A runner image provisioned with a low inode ratio runs out of inodes long before it runs out of bytes.

How to fix it

Confirm it is inodes, not bytes

Check inode usage and find the directories holding the most files.

Terminal
df -i
for d in /*; do echo "$(find "$d" -xdev 2>/dev/null | wc -l) $d"; done | sort -rn | head

Delete the file-heavy directories

Remove the caches or dependency trees holding millions of inodes.

Terminal
rm -rf node_modules ~/.cache/pip ~/.npm /tmp/*
df -i

How to prevent it

  • Clean dependency caches between jobs on reused runners.
  • Provision runner filesystems with an adequate inode ratio for file-heavy stacks.
  • Prefer package managers/caches that store fewer, larger files where possible.

Frequently asked questions

Why does df -h show free space if the disk is "full"?
There are two independent resources: data blocks (bytes) and inodes (file slots). df -h shows bytes; df -i shows inodes. You can exhaust either one. Millions of tiny files exhaust inodes while leaving bytes free.

Related guides

Tired of flaky CI? Latchkey auto-heals failed jobs and retries them for you. Start free →