# Self-Healing CI: Recovering from Spot-Instance Preemption (Exit 143)

> A job killed by spot-instance preemption (SIGTERM / exit 143) is infrastructure, not a build bug. See the manual fix and how self-healing CI retries it.

Source: https://latchkey.dev/learn/self-healing-ci/self-heal-spot-preemption-sigterm-143  
Updated: 2026-06-25

When a spot runner is reclaimed mid-job, the build did not fail - the machine was taken away. The right move is to run it again on fresh capacity.

## What makes a failure safely retryable

Automatic retry is only correct for failures that are genuinely transient. Retrying a deterministic failure wastes minutes and hides a real defect, so the classification matters more than the retry mechanism.

- Safe to retry: network timeouts, registry 5xx, transient DNS failures, a service container that was not ready, a spot instance reclaimed mid-run.
- Not safe to retry: assertion failures, compile errors, lint violations, anything that fails identically on every attempt.
- Ambiguous, and worth investigating rather than retrying: out-of-memory kills, disk exhaustion, and flaky tests. These repeat under load and a retry only hides the trend.
- Always record that a retry happened. A pipeline that silently retries is a pipeline whose real failure rate you do not know.

## FAQ

### Is exit 143 always a preemption?

Not always - 143 means the process received SIGTERM from something. On spot/preemptible infrastructure it is most often the host being reclaimed, but it can also come from a job cancellation or an external timeout. Self-healing distinguishes a host-driven termination from a deliberate cancel and only retries the former.

---

Latchkey runs CI/CD that repairs its own failures. Agent entry points: https://latchkey.dev/agent.txt, https://latchkey.dev/openapi.json, https://latchkey.dev/llms.txt