What Is Rate Limiting? Capping Request Volume
Rate limiting is a policy that caps how many requests a client may make within a time window, protecting a service from overload and abuse by rejecting excess requests.
APIs cannot serve unlimited requests, so they enforce rate limits that allow a set number of calls per window and reject the rest. Pipelines that call APIs in tight loops, or run many jobs at once against the same API, routinely bump into these limits and must handle them gracefully.
Limits per window
A rate limit defines a budget, such as requests per minute or per hour, scoped to a client, token, or IP. Once the budget is spent, further requests are rejected until the window resets.
The 429 response
When you exceed a limit, the API returns 429 Too Many Requests, often with a Retry-After header telling you how long to wait. Honoring that header is the correct way to back off.
How CI trips limits
- Many parallel jobs share one API token.
- Tight polling loops that call far too often.
- A matrix build multiplying the call count.
Handling limits gracefully
Respect Retry-After, add backoff with jitter, cache responses, and reduce unnecessary calls. Spreading requests across time keeps a pipeline under the limit instead of hammering and getting blocked.
Limits at the gateway
Rate limiting is often enforced at an API gateway in front of the service, so a 429 can come from the gateway before the request ever reaches the backend.
Transient and retryable
A 429 is explicitly retryable after the indicated wait, unlike a 4xx that signals a bad request. Latchkey runners back off and retry transient 429s from common endpoints, honoring Retry-After so brief throttling does not fail a job.
Key takeaways
- Rate limiting caps requests per window to protect a service from overload.
- Exceeding the limit returns 429, usually with a Retry-After to honor.
- A 429 is retryable with backoff, unlike a 4xx that means a bad request.