# Self-Healing CI: Recovering from a Spot Instance Interruption

> A job cut off because the spot instance was interrupted is infrastructure, not a build bug. See the manual fix and how self-healing CI reruns on fresh capacity.

Source: https://latchkey.dev/learn/self-healing-ci/self-healing-spot-instance-interrupted  
Updated: 2026-06-26

When a spot instance is reclaimed, the build did not fail -- the machine was taken away. Running it again on fresh capacity is the right move.

## What makes a failure safely retryable

Automatic retry is only correct for failures that are genuinely transient. Retrying a deterministic failure wastes minutes and hides a real defect, so the classification matters more than the retry mechanism.

- Safe to retry: network timeouts, registry 5xx, transient DNS failures, a service container that was not ready, a spot instance reclaimed mid-run.
- Not safe to retry: assertion failures, compile errors, lint violations, anything that fails identically on every attempt.
- Ambiguous, and worth investigating rather than retrying: out-of-memory kills, disk exhaustion, and flaky tests. These repeat under load and a retry only hides the trend.
- Always record that a retry happened. A pipeline that silently retries is a pipeline whose real failure rate you do not know.

## FAQ

### What causes Self-Healing CI: recovering from a spot instance interruption?

A job stops abruptly mid-run because the cloud provider reclaimed the underlying spot instance with little warning. The code is fine; the runner disappeared. A human re-runs the job on new capacity and it completes unchanged.

### How do I fix Self-Healing CI: recovering from a spot instance interruption manually?

[object Object]

### Can Self-Healing CI: recovering from a spot instance interruption be fixed automatically?

A spot interruption has an unmistakable signature -- the host is reclaimed, not an application error -- and the safe response is to retry on new capacity. A self-healing CI pipeline recognizes the interruption, reruns the job on a fresh instance, and only counts a failure that reproduces without an interruption, so a reclaimed instance

---

Latchkey runs CI/CD that repairs its own failures. Agent entry points: https://latchkey.dev/agent.txt, https://latchkey.dev/openapi.json, https://latchkey.dev/llms.txt