Kubernetes Pod Evicted "exceeds emptyDir volume sizeLimit" in CI

An emptyDir volume with a sizeLimit is evicted once its usage exceeds that limit. The pod is killed with an event noting the emptyDir grew past its allowed size - the scratch space filled up.

What this error means

A pod is evicted partway through a job; kubectl describe pod (or the events) report the emptyDir volume exceeded its sizeLimit. It reproduces whenever the workload writes more scratch data than the limit allows.

kubectl describe pod

Status:   Failed
Reason:   Evicted
Message:  Usage of EmptyDir volume "scratch" exceeds the limit "1Gi".

Common causes

Scratch data exceeds the sizeLimit

A build cache, download, or temp output written to the emptyDir grows beyond the configured sizeLimit, triggering eviction.

medium: Memory emptyDir counts against memory

An emptyDir with medium: Memory is a tmpfs that consumes node memory and is bounded by sizeLimit (and the container memory limit). Filling it can evict the pod or OOM it.

How to fix it

Right-size the limit to the real scratch need

Measure peak scratch usage and set sizeLimit above it, or remove the limit if the node has the headroom.

pod.yaml

volumes:
  - name: scratch
    emptyDir:
      sizeLimit: 5Gi   # was 1Gi; cover peak scratch usage

Reduce or relocate the scratch data

Clean up intermediate files as the job runs so peak usage stays low.
Stream large downloads instead of staging the whole file on disk.
For big persistent scratch, use a PVC instead of an emptyDir.

How to prevent it

Set emptyDir.sizeLimit based on measured peak scratch usage with headroom.
Clean intermediate files during long jobs to cap usage.
Use a PVC, not emptyDir, for large or long-lived scratch data.