Skip to content
Latchkey

Helm "release failed, rolling back" - Fix --atomic Rollbacks in CI

helm upgrade --atomic rolls the release back to its previous revision if the upgrade does not succeed within the timeout. Seeing a rollback message means the upgrade itself failed - the rollback is the cleanup, not the root cause.

What this error means

helm upgrade --atomic exits non-zero with UPGRADE FAILED: ... ; release <name> has been rolled back to revision N. The cluster is back on the old revision, and CI fails. The real error is whatever made the upgrade not become ready.

helm output
Error: UPGRADE FAILED: timed out waiting for the condition
Error: release api has been rolled back to revision 7

Common causes

New release never became ready in time

With --atomic --wait, Helm waits for resources to be ready; a crashing pod, failing probe, or unschedulable workload means it times out and rolls back.

A resource in the upgrade was rejected

An invalid manifest, an immutable-field change, or an admission denial fails the upgrade, triggering the atomic rollback.

How to fix it

Find why the upgrade failed before the rollback

The rollback hides the symptom. Re-run without --atomic (or inspect history and the workloads) to see the real failure.

Terminal
helm history api
helm upgrade api ./chart --wait --timeout 5m   # no --atomic, to observe the failure
kubectl get pods -l app.kubernetes.io/instance=api
kubectl describe deploy -l app.kubernetes.io/instance=api | sed -n '/Conditions/,/Events/p'

Fix the readiness/resource problem, then re-enable --atomic

  1. If it timed out, fix what kept pods from becoming ready (image, probe, scheduling) or raise --timeout for a genuinely slow rollout.
  2. If a resource was rejected, correct the manifest/immutable change and re-apply.
  3. Keep --atomic for safe deploys once the underlying upgrade succeeds reliably.

How to prevent it

  • Validate the chart against the cluster (--dry-run) before the atomic upgrade.
  • Set --timeout to the app’s real readiness time so healthy-but-slow rollouts are not rolled back.
  • Gate CI on the upgrade and surface workload events when it fails.

Related guides

Tired of flaky CI? Latchkey auto-heals failed jobs and retries them for you. Start free →