pip ReadTimeoutError and 429 from PyPI in CI
A pip ReadTimeoutError against PyPI in CI means the connection opened and then went quiet: the host accepted the request and sent nothing back inside the socket timeout. On GitHub Actions the fix is to raise the timeout the failing pip actually reads and to cache the wheels, because the default is 15 seconds and a cold job downloads everything again.
What this error means
The install step runs for a while and then ends in a Python traceback, and the line that matters is near the bottom, in the shape "ReadTimeoutError: HTTPSConnectionPool(host='pypi.org', port=443): Read timed out." The host in it tells you which half of the job failed, and the recorded run below names the other one. pypi.org is the index, which pip queries to resolve a version; files.pythonhosted.org is where the wheel itself lives, and a stall there happens with the resolution already done. A second shape is the throttle: pip abandons the download and reports a 429 status for a files.pythonhosted.org URL, which is the same interruption with a different reason. A third is a run of "Retrying (Retry(total=N ...)) after connection broken by" warnings, which is pip narrating its own attempts before it decides. In the run below the download host is mapped to a TLS server on loopback that completes the handshake and then never answers, because a stalled download from the real host cannot be produced on demand; the index lookup, the certificate check and the traceback are pip's own. It also sets the timeout to 5 seconds rather than pip's default of 15, so the stall turns terminal in seconds instead of a minute, which is why the evidence below reads read timeout=5.0.
pip._vendor.requests.exceptions.ConnectionError: HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Max retries exceeded with url: /packages/f9/9b/335f9764261e915ed497fcdeb11df5dfd6f7bf257d4a6a2a686d80da4d54/requests-2.32.3-py3-none-any.whl.metadata (Caused by ReadTimeoutError("HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Read timed out. (read timeout=5.0)"))Reproduced on a Latchkey runner
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/pip/_vendor/requests/sessions.py", line 602, in get
return self.request("GET", url, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/pip/_internal/network/session.py", line 520, in request
return super().request(method, url, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/pip/_vendor/requests/sessions.py", line 589, in request
resp = self.send(prep, **send_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/pip/_vendor/requests/sessions.py", line 703, in send
r = adapter.send(request, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/pip/_vendor/cachecontrol/adapter.py", line 76, in send
resp = super().send(request, stream, timeout, verify, cert, proxies)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/pip/_vendor/requests/adapters.py", line 519, in send
raise ConnectionError(e, request=request)
pip._vendor.requests.exceptions.ConnectionError: HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Max retries exceeded with url: /packages/f9/9b/335f9764261e915ed497fcdeb11df5dfd6f7bf257d4a6a2a686d80da4d54/requests-2.32.3-py3-none-any.whl.metadata (Caused by ReadTimeoutError("HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Read timed out. (read timeout=5.0)"))
[latchkey-bash-wrapper] BEGIN sidecar POST (boot_wait=30s max_time=320s url=http://localhost/diagnose socket=/run/latchkey-self-heal/sock)
[latchkey-bash-wrapper] END sidecar POST ok (attempts=1 http=200)
attempt 2, pip 24.0, PIP_DEFAULT_TIMEOUT=100 PIP_RETRIES=5
files.pythonhosted.org in /etc/hosts: 0
Collecting requests==2.32.3
Downloading requests-2.32.3-py3-none-any.whl.metadata (4.6 kB)
Downloading requests-2.32.3-py3-none-any.whl (64 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 64.9/64.9 kB 5.3 MB/s eta 0:00:00
Saved /home/runner/wheels/requests-2.32.3-py3-none-any.whl
Successfully downloaded requests
downloaded requests-2.32.3-py3-none-any.whlSet PIP_DEFAULT_TIMEOUT + a PyPI index fallback and retried pip with exponential backoff
pip has one timeout, and it is a socket timeout
--timeout is documented as "set the socket timeout", and it defaults to 15 seconds. It is not a budget for the whole download: it is how long pip will wait for the next piece of data to arrive. A large wheel on a slow link does not trip it, and a host that goes quiet for sixteen seconds does.
--retries defaults to 5 and is documented as the maximum attempts to establish a new HTTP connection, though the run below ends in "Max retries exceeded" with a read timeout as its cause, so the budget covers more than the name suggests. Raise both, and raise the timeout first.
env:
PIP_DEFAULT_TIMEOUT: '60'
PIP_RETRIES: '5'Common causes
The download host went quiet part way through
The ordinary case. A CDN edge accepts the connection, serves some of the wheel and then stops, and pip waits 15 seconds before it calls that a failure. Nothing about the package or the pin is involved, which is why the same requirements file passes on the next run.
You raised the timeout and the failing pip never read it
The wasted fix on this failure, and an easy one to make. pip install --timeout 60 sets the timeout for that one invocation, and a build has more than one: tox, Poetry, a Dockerfile layer and a build backend each start their own pip. An environment variable or a pip.conf reaches all of them, a flag on one line reaches one of them, and a flag beats the environment wherever both apply.
The index throttled the run
A 429 from a PyPI host means the request rate from this address was too high. On hosted runners the address is shared, so a wide matrix, a monorepo installing the same requirements in every leg, or simply a busy hour is enough. It is transient and it is also a signal that the job is asking for more than it needs.
The index is not PyPI
A private index, a corporate proxy or a regional mirror sits between pip and the packages, with its own capacity and its own timeouts. In our experience a job that started timing out the week a proxy appeared is not seeing a PyPI problem, and the host in the traceback says so.
How to fix it
Raise the timeout through the environment
- Set
PIP_DEFAULT_TIMEOUTandPIP_RETRIESin the job'senv:block so every pip in the job inherits them. - Start at 60 seconds rather than 15; the failure is a host going quiet, so the fix is patience.
- Leave the command lines alone, because a flag on one command overrides the environment for that command only.
jobs:
test:
runs-on: ubuntu-latest
env:
PIP_DEFAULT_TIMEOUT: '60'
PIP_RETRIES: '5'
steps:
- uses: actions/checkout@v7
- uses: actions/setup-python@v7
with:
python-version: '3.12'
cache: pip
- run: pip install -r requirements.txtPut the setting where the nested pip will find it
A Dockerfile layer, a tox environment and a Poetry install all start a pip that reads its own configuration. Write a pip.conf into the image, or set the same environment variables with ENV, so the setting survives the layer boundary.
# Dockerfile
ENV PIP_DEFAULT_TIMEOUT=60 PIP_RETRIES=5
RUN pip install --no-cache-dir -r requirements.txtWait longer on a throttle, and ask for less
A rate limit needs a minute, not a second. Back off before retrying, and then reduce the request count: a restored pip cache, a wheelhouse committed to an artifact, or a private index you control all cut the number of times the job asks a public host for anything.
- name: Build a wheelhouse once
run: pip wheel --wheel-dir wheelhouse -r requirements.txt
- name: Install from it
run: pip install --no-index --find-links wheelhouse -r requirements.txtRead the host before you change anything
The host in the traceback is the fastest diagnosis on the page. A private index host means the problem is yours, a PyPI host means it is transient, and a mirror that is neither means somebody added a proxy. Print the effective index once and the guessing stops.
pip config list
python -m pip config debug | head -20The throttle is a different failure with the same cure
A 429 from a package index is not a stall; it is an answer, and one that arrives quickly. It means this client asked for too much too fast, and on a hosted runner the client is an address shared with every other job on the same egress. Waiting is the only correct first move, and for longer than you would wait on a 5xx, because a rate limit window recharges on a clock.
What actually removes it is asking for less. A cached wheel is not a request, and a wheelhouse built once and restored on every run turns a hundred requests into zero.
Cache the wheels, not the virtual environment
Caching ~/.cache/pip keeps the wheels pip already downloaded, so a repeat install resolves from disk. setup-python manages it for you from the requirements file.
Caching the virtual environment instead ages badly: it captures a resolved tree rather than the inputs, so one changed pin leaves a cache that is wrong rather than stale. Keep the store, rebuild the environment.
- uses: actions/setup-python@v7
with:
python-version: "3.12"
cache: pip
cache-dependency-path: requirements.txtWhat the runner does about it
Latchkey detects the stall as PIP_INDEX_TIMEOUT at confidence 0.96, as high as anything in the network set, because the frame it matches is unambiguous: the vendored urllib3 timeout against a PyPI host, inside a traceback pip only prints as it exits. Its plan is not a bare retry. It sets PIP_DEFAULT_TIMEOUT to 100 and PIP_RETRIES to 5 and then retries the step, which is why the recorded run above shows those two values on the second attempt and a saved wheel under them.
Two neighbours cover the rest of the family. PYPI_RATE_LIMIT fires on a PyPI-host 429 at 0.92 and waits 60 seconds before the first retry rather than 2. PIP_NETWORK_RETRY fires at 0.90 on the urllib3 retry warnings pip prints while it is still trying. All three are live, and all three end in a retried step.
How to prevent it
- Set
PIP_DEFAULT_TIMEOUTandPIP_RETRIESin the workflow environment, not per command. - Cache the pip store with
setup-python, so most installs never reach the network. - Pin requirements so the download set is stable and the cache key is meaningful.
- Keep one index configured per job, and print it when a run fails.
Frequently asked questions
What is a good pip timeout for CI?
Why does pip print Retrying after connection broken?
Does caching the pip cache stop PyPI timeouts?
~/.cache/pip is never requested again. It does not stop the index lookup, so a cold resolution still talks to the index. Pinning requirements and caching the store turn a hundred requests into a handful.