Skip to content
LatchkeyLatchkey home

Self-Healing CI: Auto-Retrying an Azure Blob Storage Throttle

A ServerBusy throttle from Azure Blob Storage is a temporary rate limit, not a build error -- the same request succeeds after a short backoff.

The problem

An artifact transfer to/from Azure Blob Storage fails because the service returned a 503 ServerBusy (or timed out) under load. The build and the blob are fine; the request was briefly throttled. A human re-runs and the transfer completes unchanged.

Typical symptom
Status 503: The server is busy. ErrorCode: ServerBusy
RequestId: ... (retry the request after a short delay)

Why it happens

Azure Storage throttles requests with a 503 ServerBusy when a partition or account briefly exceeds its target request rate, so a burst of artifact transfers can be throttled even though every request is valid.

The throttle clears as the request rate falls back under target, so a ServerBusy now succeeds once the call is retried with backoff, with no change to the blob.

The manual fix

Manual mitigations for an Azure Blob throttle:

  1. Re-run the job to retry the transfer.
  2. Add exponential backoff with jitter and rely on the SDK’s built-in retry policy.
  3. Reduce concurrency and spread load to stay under the request-rate target.

How this gets automated

An Azure ServerBusy throttle carries an unmistakable transient signature and a safe default action: back off and retry. A self-healing CI pipeline recognizes the throttle, waits with backoff, retries the transfer, and only escalates if the operation keeps failing after the rate should have eased, distinguishing a throttle from a genuine storage or permissions error.

Frequently asked questions

What causes Self-Healing CI: Auto-Retrying an Azure blob storage throttle?
An artifact transfer to/from Azure Blob Storage fails because the service returned a 503 ServerBusy (or timed out) under load. The build and the blob are fine; the request was briefly throttled. A human re-runs and the transfer completes unchanged.
How do I fix Self-Healing CI: Auto-Retrying an Azure blob storage throttle manually?
[object Object]
Can Self-Healing CI: Auto-Retrying an Azure blob storage throttle be fixed automatically?
An Azure ServerBusy throttle carries an unmistakable transient signature and a safe default action: back off and retry. A self-healing CI pipeline recognizes the throttle, waits with backoff, retries the transfer, and only escalates if the operation keeps failing after the rate should have eased, distinguishing a throttle from a genuine

Related guides

References

Latchkey auto-heals failures like this one - detected, fixed, and retried without you. Start free → 30-day trial · No credit card