Self-Healing CI: Recovering a 5xx Mid-Pull from an Image Registry

A registry that 5xxes partway through a pull had a momentary backend error, not a corrupt image -- retrying resumes or restarts the pull and it completes.

The problem

An image pull fails because the registry returned a 5xx (500/502/503) while streaming a layer -- the pull started fine and then a backend error interrupted it mid-transfer. The image and tag are valid; the registry blipped. A human re-runs the job and the pull completes unchanged.

Typical symptom

failed to copy: httpReadSeeker: failed open: unexpected status code ... 502 Bad Gateway
# or
error pulling image: received unexpected HTTP status: 500 Internal Server Error

Why it happens

Pulling an image streams several layers from a registry backend over the course of the transfer. A momentary backend error or overload partway through returns a 5xx on one layer, interrupting the pull even though the image is intact and most layers came down fine.

It is a transient backend error, not a bad image: the same pull resumes or restarts cleanly on a retry, often reusing the layers already fetched, with no change to the image.

The manual fix

Manual mitigations for a 5xx mid-pull:

Re-run the job to retry the pull (already-fetched layers are typically reused).
Add retry-with-backoff around the pull command.
Pull through a registry mirror or pull-through cache to reduce direct dependence on the backend.

How this gets automated

A 5xx mid-pull has a recognizable transient signature -- a server error on one layer of an otherwise-healthy transfer -- and the safe response is to retry the pull with backoff. A self-healing CI pipeline detects the interrupted pull, retries it (resuming cached layers where possible), and only escalates if the image stays unpullable across retries, distinguishing a backend blip from a genuinely missing or broken image.