Python "IOError: [Errno 28] No space left on device" in CI

Python raised [Errno 28] No space left on device - the POSIX ENOSPC errno surfaced as an OSError/IOError. A write to the full runner disk was refused by the kernel.

What this error means

A Python step fails with OSError: [Errno 28] No space left on device while writing a file, a cache, a model checkpoint, or a temp file. The disk is full; df -h confirms 100%. Cleanup or a bigger disk fixes it with no code change.

OSError: [Errno 28] No space left on device
# during e.g. open(path, 'w').write(...), shutil.copy, or torch.save(...)

Common causes

The runner disk is full

Caches (~/.cache/pip), datasets, checkpoints, and build output filled the filesystem. The next write - wherever it lands - fails with errno 28.

A large dataset or model artifact exceeded the disk

Data-heavy Python jobs (ML training, large downloads) can write more than a small runner disk holds within a single step.

How to fix it

Check space and the largest paths

df -h
du -xh ~ /tmp 2>/dev/null | sort -rh | head

Reclaim space or relocate large writes

1. Clear caches (pip cache purge, rm -rf ~/.cache/*) and old artifacts.
2. Write large outputs (checkpoints, datasets) to a larger scratch volume.
3. Stream/process data instead of materializing it all on disk.

How to prevent it

• Clean caches between jobs on reused runners.
• Point large-output paths at a roomy scratch volume.
• Use larger-disk runners for data- or model-heavy pipelines.

Related guides