How to Run Great Expectations in CI

Great Expectations validates data, so CI needs the datasource extras for your backend plus a reachable test dataset to run checkpoints against.

The great_expectations package installs cleanly, but each datasource (Postgres, Spark, BigQuery) needs its own extras and driver. CI failures are usually a missing datasource dependency or an unreachable database.

Why it fails in CI

The SQL/Spark datasource extras are not installed → datasource init fails.
The configured database is not reachable in CI.
Data Docs or checkpoint config paths differ between local and CI.

Install and run it reliably

Install GX with the extras for your datasource, point it at a test database via env, and run the checkpoint headlessly.

Terminal

# install with the datasource extras you use
pip install 'great_expectations[postgresql]'

export GX_DB_URL=postgresql://user:pass@localhost:5432/app

# run a checkpoint non-interactively
great_expectations checkpoint run my_checkpoint

Cache & speed

Cache ~/.cache/pip keyed on requirements. Use a database service container for SQL datasources and gate the checkpoint on its health.

.github/workflows/ci.yml

- uses: actions/cache@v4
  with:
    path: ~/.cache/pip
    key: pip-${{ runner.os }}-${{ hashFiles('**/requirements*.txt') }}

Common errors

Could not load datasource ... missing dependency → install the matching [extras].
Unable to connect to ... database → start a DB service and wait for health.
Checkpoint ... not found → keep the GX config in the repo so CI sees it.

Key takeaways

Install Great Expectations with the datasource extras you use.
Provide a reachable test database via env and a service container.
Run checkpoints non-interactively in CI.