Docker overlay2 "no space left on device" From Inode Exhaustion in CI

Docker reports "no space left on device" even though df -h shows free space. The overlay2 storage driver creates huge numbers of small files, and the filesystem under /var/lib/docker exhausted its inodes, not its bytes.

What this error means

A pull, build, or container start fails with no space left on device, but df -h shows plenty of free disk. df -i reveals 100% inode usage on the filesystem holding /var/lib/docker.

docker / df -i output

failed to register layer: mkdir /var/lib/docker/overlay2/.../diff:
no space left on device
# yet: df -h shows 40% used, while df -i shows 100% inodes used

Common causes

Inodes exhausted by many small layer files

overlay2 stores each layer as a directory full of small files. Accumulated images, build cache, and containers can use up all inodes long before bytes run out.

A small-inode filesystem on the graph root

A filesystem formatted with few inodes (or a small partition for /var/lib/docker) hits the inode cap quickly on an image-heavy runner.

How to fix it

Confirm inodes, then prune

Check inode usage and reclaim by pruning unused Docker data.

Terminal

df -i /var/lib/docker
docker system prune --all --force --volumes
docker builder prune --all --force

Give the graph root more inodes

Mount /var/lib/docker on a filesystem formatted with more inodes.
Prune images/build cache at the start of each job on long-lived runners.
Avoid images with pathologically many tiny files where possible.

How to prevent it

Monitor df -i, not just df -h, on Docker runners.
Prune Docker data regularly on long-lived/self-hosted runners.
Provision the graph-root filesystem with adequate inodes.