CI/CD for a Flask + Celery App with GitHub Actions

Lint, test the Flask app and Celery tasks against Redis and Postgres, then build a Docker image.

This recipe tests a Flask app whose Celery workers use Redis as the broker and Postgres for data. CI runs tests against both service containers so task behavior is exercised end to end, then builds an image.

What the pipeline does

• install deps from requirements
• lint with ruff
• start Redis and Postgres service containers
• run pytest including Celery task tests
• build and push a Docker image

The workflow

Redis backs the Celery broker and Postgres backs the app. CELERY_BROKER_URL and DATABASE_URL point at the services; run Celery in eager mode or against the broker for task tests.

name: CI
on:
  push:
    branches: [main]
  pull_request:
jobs:
  test:
    runs-on: ubuntu-latest
    services:
      redis:
        image: redis:7
        ports: ["6379:6379"]
        options: >-
          --health-cmd "redis-cli ping" --health-interval 10s
          --health-timeout 5s --health-retries 5
      postgres:
        image: postgres:16
        env:
          POSTGRES_PASSWORD: postgres
          POSTGRES_DB: app
        ports: ["5432:5432"]
        options: >-
          --health-cmd "pg_isready" --health-interval 10s
          --health-timeout 5s --health-retries 5
    env:
      CELERY_BROKER_URL: redis://localhost:6379/0
      DATABASE_URL: postgresql://postgres:postgres@localhost:5432/app
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
          cache: pip
      - run: pip install -r requirements.txt -r requirements-dev.txt
      - run: ruff check .
      - run: pytest

Caching and speed

setup-python with cache: pip restores the wheel cache keyed on requirements files. Two service containers plus task tests make this run heavy; cheaper managed runners such as Latchkey (around 69% cheaper than GitHub-hosted) keep it fast and auto-retry a flaky Redis or Postgres pull.

Deploying

Build one image and run it as both the web process (gunicorn) and the worker process (celery -A app worker) with different commands. Add a deploy job that builds and pushes the image, then rolls out web and worker services once tests pass.

Key takeaways

• Back the Celery broker with a Redis service and the app with Postgres.
• Run task tests against the broker or in eager mode for determinism.
• Ship one image and vary the start command for web vs worker.

Related guides