Docker net/http: TLS handshake timeout in CI
A Docker TLS handshake timeout in CI means the pull opened a connection to the registry and then ran out its deadline before the certificate exchange finished. It is a path problem rather than an image problem, which is why the same docker pull works on the next run, and why the first thing worth changing is how many pulls your job starts at once.
What this error means
The pull fails in seconds rather than minutes, with a single line from the daemon quoting the URL it was trying and a Go networking error. Three wordings mean the same thing at three different points: a handshake that started and did not finish, a request canceled while waiting for headers, and a context deadline exceeded. All three are the client giving up rather than the registry refusing you, which is the difference between this and a rate limit or an authentication failure. This page carries no recorded run. The reproduction available to us is a listener that accepts the connection and never completes the handshake, which produces the message but models a registry that is permanently unreachable rather than the transient this failure actually is, so the block below is the sample text the Latchkey pattern library matches on, labeled as such, and the causes come from the daemon issues in the references.
Error response from daemon: Get "https://registry-1.docker.io/v2/": net/http: TLS handshake timeout
Error response from daemon: Get "https://registry-1.docker.io/v2/": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
Error response from daemon: failed to resolve reference "docker.io/library/node:lts-slim": failed to do request: Head "https://registry-1.docker.io/v2/library/node/manifests/lts-slim": net/http: TLS handshake timeoutRead which stage the pull died in
A pull is four things in a row, and the daemon prints a different error for each. Sorting yours before changing anything saves an afternoon of tuning a layer that was never involved.
| What the daemon prints | Stage that failed | Where to look |
|---|---|---|
no such host, server misbehaving | Name resolution | The resolver, not the registry |
connection refused, i/o timeout on dial | TCP connect | Egress rules, a firewall, an unreachable address |
net/http: TLS handshake timeout | TLS handshake | Saturated egress, packet loss, a TLS inspection box |
Client.Timeout exceeded while awaiting headers | The request after the handshake | A registry that is slow rather than down |
429 Too Many Requests, toomanyrequests | The registry answering | A pull limit, which has its own page |
Common causes
The runner was pulling several things at once
The most common and the most fixable. Handshakes are latency sensitive and small; a link busy moving layer blobs will delay them enough to trip a deadline measured in seconds. A job that fails in the step where compose starts six containers, and passes when it pulls one image, has said what the problem is.
The registry edge was having a bad minute
Public registries are shared infrastructure and they have slow periods. The moby issue tracker carries this one continuously, with the same shape each time: intermittent, not reproducible on demand, gone on the next attempt. Nothing in your workflow caused it and nothing in your workflow will prevent it, so the answer is a retry and a mirror.
Something is inspecting TLS between the runner and the registry
A corporate proxy, an egress appliance or a service mesh sidecar terminating TLS adds work and latency to every handshake, and sometimes rewrites the certificate chain. If the failures are constant rather than occasional, and only from one network, this is the first place to look rather than the last.
The path itself is lossy, or IPv6 is a black hole
Packet loss and a broken path MTU both show up as handshakes that stall, because the handshake needs several round trips before anything useful has happened. An environment that advertises IPv6 without a working route is the same failure with a cleaner cause: the daemon tries the address it was given, waits, and reports a timeout.
How to fix it
Retry the pull, with a bounded loop
- Pull explicitly before the step that needs the image, so a failure is attributed to the pull rather than to a build or a test.
- Retry three times with a short backoff. A handshake that fails twice in a row is usually a path problem rather than a busy minute.
- Fail the step on the last attempt rather than continuing into a build that has no image.
for attempt in 1 2 3; do
docker pull "$IMAGE" && break
echo "pull attempt $attempt failed, retrying"
sleep $((attempt * 5))
done
docker image inspect "$IMAGE" --format "{{.Id}}"Lower how many downloads the daemon runs at once
Set max-concurrent-downloads in the daemon configuration and restart the daemon. Three is a reasonable starting point on a small runner, and it is the change most likely to remove the failure outright when several pulls overlap.
echo '{"max-concurrent-downloads": 3}' | sudo tee /etc/docker/daemon.json
sudo systemctl restart dockerPut a mirror or a cache in front of the registry
A pull that ends inside your own network cannot time out on somebody else's edge. A pull-through cache also takes the public registry out of the picture for its rate limits, which is a second problem solved by the same change, and it is covered in detail on the registry mirror page.
- uses: docker/setup-buildx-action@v4
with:
driver-opts: |
image=moby/buildkit:latest
network=host
buildkitd-config-inline: |
[registry."docker.io"]
mirrors = ["mirror.example.com"]Prove which stage is failing before tuning anything else
One command separates a name resolution problem, a connectivity problem and a handshake problem, and it takes two seconds. Run it in the same job, in the same network, at the point the pull fails.
getent hosts registry-1.docker.io
echo | time openssl s_client -connect registry-1.docker.io:443 \
-servername registry-1.docker.io 2>&1 | head -3Concurrency is the lever that usually moves this
A handshake times out because the packets that carry it were slow, lost or queued, and the commonest reason on a runner is that the machine is doing several other pulls at the same time. A compose file bringing up six services, a matrix of jobs behind one address, and a build that pulls three base images all push against the same egress.
The daemon takes a limit for this directly, and lowering it is a smaller change than it sounds: layers still download in parallel, just fewer at a time, so a saturated link stops starving the handshakes that are trying to start.
{
"max-concurrent-downloads": 3,
"registry-mirrors": ["https://mirror.example.com"]
}What a runner does about it
Latchkey runs GitHub Actions jobs on managed runners that read a failed step's output and act on it, and a pull that died in the handshake is the kind of failure that is worth acting on: it is transient, it is nothing to do with the code under test, and the next attempt usually succeeds. This page names no pattern and claims no repair, because both belong with a recorded run and this page does not have one. See how self-healing works for what the runner does with a failing step.
How to prevent it
- Pull images explicitly, early, and with a bounded retry, rather than inside the step that needs them.
- Keep
max-concurrent-downloadslow on small runners that pull several images per job. - Serve base images from a mirror or a registry inside your own network.
- Pin image tags to digests so a retry pulls the same bytes, and a cached layer is reused rather than refetched.
Frequently asked questions
What does net/http: TLS handshake timeout mean in docker pull?
Why does docker pull fail with TLS handshake timeout only sometimes?
Does a registry mirror fix TLS handshake timeouts?
Do proxy settings in my workflow apply to docker pull?
/etc/systemd/system/docker.service.d/, followed by a daemon restart.