Self-Healing CI: Auto-Retrying a Flaky Network Download
A file fetch that fails partway is a momentary network problem, not a missing file -- the same URL almost always downloads cleanly on a retry.
The problem
A step that downloads an installer, binary, or asset over HTTP fails with a partial transfer, a reset connection, or a non-zero curl/wget exit. The URL and the file are fine; the transfer hit a brief network blip. A human re-runs the job and the download completes unchanged.
curl: (18) transfer closed with outstanding read data remaining
# or
wget: ... Read error (Connection reset by peer) in headers.Why it happens
A single download moves bytes over one connection that can be interrupted by congestion, an upstream hiccup, or a server-side timeout, leaving a truncated file or a hard error even though the asset is intact at the source.
A bare download with no retry turns one unlucky moment into a failed pipeline, because the step has no second chance to complete the transfer.
The manual fix
Manual mitigations for a flaky download:
- Re-run the job to retry the download.
- Use the client’s built-in retry flags (e.g. curl --retry, wget --tries) with backoff.
- Verify the file with a checksum and re-download if it does not match.
curl --retry 5 --retry-delay 2 --retry-all-errors -fL -o tool.tgz "${URL}"
echo "${EXPECTED_SHA} tool.tgz" | sha256sum -c -How this gets automated
A failed download has a recognizable transient signature -- a partial or reset transfer -- and the safe response is to retry with backoff. A self-healing CI pipeline detects the download failure, retries the fetch, and only escalates if the asset is genuinely unreachable across retries, distinguishing a momentary blip from a real missing or moved file.