Skip to content
Latchkey

Kubernetes "Back-off restarting failed container" - Fix in CI

"Back-off restarting failed container" is the kubelet event that accompanies CrashLoopBackOff. It is not a separate failure - it tells you the container exited and the kubelet is deliberately waiting before the next restart attempt.

What this error means

kubectl describe pod shows a repeating Warning BackOff ... Back-off restarting failed container event under Events, with the pod stuck out of Ready. The interval between restarts grows (10s, 20s, 40s … up to 5m).

kubectl describe pod
Events:
  Type     Reason   Age                 From     Message
  Warning  BackOff  12s (x6 over 2m)    kubelet  Back-off restarting failed
  container api in pod api-6c8f9d5b7c-h2k4p

Common causes

The container keeps exiting

The event only appears because the container terminated and was restarted repeatedly. The root cause is whatever makes the process exit - read its logs and exit code, not the back-off event.

Probe-driven restarts

A failing liveness probe kills the container on a schedule, so the kubelet restarts it and emits the back-off event even if the app itself "runs".

How to fix it

Find the real exit reason behind the event

The BackOff event is downstream. Pull the previous container logs and the termination state to see why it exited.

Terminal
kubectl logs <pod> --previous
kubectl get pod <pod> -o jsonpath='{.status.containerStatuses[0].lastState.terminated}'; echo

Separate a crash from a probe kill

  1. If lastState.terminated.reason is Error/OOMKilled, the app crashed - fix the app or limits.
  2. If the container is killed on a regular interval, suspect a failing liveness probe and check its config.
  3. Re-deploy once the underlying cause is fixed; the back-off clears on a healthy run.

How to prevent it

  • Treat the BackOff event as a pointer to the container’s exit reason, not the bug itself.
  • Tune liveness probes so a slow-but-healthy app is not killed and restarted.
  • Fail CI on a non-Ready rollout so back-off loops do not ship.

Related guides

Tired of flaky CI? Latchkey auto-heals failed jobs and retries them for you. Start free →