Hugging Face "429 Too Many Requests" rate limit in CI

The Hub returned 429 because too many requests came from the runner in a short window. Anonymous traffic and many parallel jobs hitting the Hub uncached are the usual triggers; authenticating and caching cut the request volume.

What this error means

A download fails or retries with "429 Client Error: Too Many Requests for url: https://huggingface.co/...", often on shared CI IPs or matrix builds.

huggingface_hub

huggingface_hub.utils._errors.HfHubHTTPError: 429 Client Error: Too Many Requests
for url: https://huggingface.co/bert-base-uncased/resolve/main/model.safetensors
(Request ID: Root=1-...)

Common causes

Anonymous requests share a lower limit

Without a token, requests fall under a stricter shared quota that many CI runners on the same egress IP exhaust quickly.

Uncached matrix jobs each re-download

Every matrix leg fetching the same large model multiplies requests and trips the limit.

How to fix it

Authenticate and cache downloads

Set HF_TOKEN so requests count against your account, and persist HF_HOME so repeats hit the cache instead of the Hub.

.github/workflows/ci.yml

env:
  HF_TOKEN: ${{ secrets.HF_TOKEN }}
  HF_HOME: ${{ github.workspace }}/.hf-cache

Back off and retry on 429

When a fetch is unavoidable, retry with exponential backoff rather than hammering the endpoint.

Terminal

export HF_HUB_DOWNLOAD_TIMEOUT=60
python -c "from huggingface_hub import snapshot_download; snapshot_download('bert-base-uncased')"

How to prevent it

Always send a token so requests use your account quota.
Cache HF_HOME so repeated jobs do not re-request.
Prefetch once, then have matrix legs read the shared cache.