Skip to content
Latchkey

ARC "failed to delete ephemeral runner" / stuck finalizer in CI

The controller tried to remove an EphemeralRunner but the delete did not complete. A finalizer runs cleanup (unregistering the runner from GitHub) and cannot finish, so the object hangs in Terminating.

What this error means

An EphemeralRunner stays in Terminating and kubectl delete hangs. The controller logs "failed to delete ephemeral runner" and the runner may still show as registered in GitHub.

controller
ERROR failed to delete ephemeral runner
  {"error": "runner is still registered with the service and could not
  be removed", "ephemeralrunner": "arc-runner-set-xxxx"}

Common causes

Cleanup cannot unregister the runner

The finalizer calls GitHub to unregister the runner; if that call fails on auth or network, deletion cannot complete.

A busy or orphaned runner still registered

A runner mid-job or orphaned by a crashed listener stays registered, blocking the finalizer.

How to fix it

Fix the cause, then let the finalizer complete

  1. Read the controller logs for why unregistration failed (auth, network).
  2. Restore the credential or egress so the finalizer can call GitHub.
  3. If the object must be force-removed, clear its finalizer as a last resort.
Terminal
kubectl patch ephemeralrunner -n arc-runners <name> \
  --type=json -p='[{"op":"remove","path":"/metadata/finalizers"}]'

Remove a stale runner in GitHub

If a runner is still listed as registered after the pod is gone, delete it from the org or repo runners page so state matches the cluster.

How to prevent it

  • Keep credentials and egress valid so cleanup can unregister runners.
  • Let listeners shut down cleanly to avoid orphaned runners.
  • Only strip finalizers manually as a last resort after fixing the cause.

Related guides

Tired of flaky CI? Latchkey auto-heals failed jobs and retries them for you. Start free →