Celery "WorkerLostError: Worker exited prematurely" in CI

Celery raises WorkerLostError when a worker child process dies while running a task. The most common cause in CI is the OS OOM killer sending SIGKILL (signal 9) because the task exhausted the runner's memory.

What this error means

A task fails with "billiard.exceptions.WorkerLostError: Worker exited prematurely: signal 9 (SIGKILL). Job: N." and the worker pool replaces the child.

billiard.exceptions.WorkerLostError: Worker exited prematurely: signal 9 (SIGKILL)
Job: 2.

Common causes

The task exceeded available memory

A task loaded too much data and the runner OOM killer sent SIGKILL, so the parent reports the child as lost.

A hard time limit or external kill

A task_time_limit SIGKILL or an external signal terminated the child mid-task.

How to fix it

Reduce memory and cap concurrency

1. Process data in batches instead of loading it all at once.
2. Lower worker concurrency so fewer tasks run in parallel on a constrained runner.
3. Recycle workers after N tasks to release leaked memory.

celery -A app worker --concurrency=2 --max-tasks-per-child=50

Set sane time limits

Use a soft time limit so a long task raises a catchable exception before a hard SIGKILL.

app.conf.task_soft_time_limit = 60
app.conf.task_time_limit = 90

How to prevent it

• Batch large data so a task fits in runner memory.
• Cap concurrency and recycle children with max-tasks-per-child.
• Prefer soft time limits so tasks fail cleanly before a kill.

Related guides