How to Run dbt in CI
dbt in CI is mostly about installing the correct adapter and feeding warehouse credentials through environment variables, not the dbt core package.
You install dbt-core plus an adapter (dbt-postgres, dbt-snowflake, dbt-bigquery). CI then needs a profiles.yml that reads credentials from env, and a reachable warehouse for build/test.
Why it fails in CI
- Only
dbt-coreis installed, not the warehouse adapter → "Could not find adapter". - profiles.yml hardcodes secrets or is missing → connection/auth errors.
dbt depspackages were not installed beforedbt build.
Install and run it reliably
Install the adapter, point dbt at a profiles.yml that reads env vars, run deps, then build. Keep credentials in CI secrets.
Terminal
pip install dbt-core dbt-postgres # swap adapter as needed
export DBT_HOST=... DBT_USER=... DBT_PASSWORD=... # from CI secrets
dbt deps
dbt build --profiles-dir ./ciCache & speed
Cache ~/.cache/pip for the adapter and the dbt packages directory (dbt_packages) so dbt deps is not re-fetched. Use a warehouse service container for Postgres-based projects.
.github/workflows/ci.yml
- uses: actions/cache@v4
with:
path: ~/.cache/pip
key: pip-${{ runner.os }}-${{ hashFiles('**/requirements*.txt') }}Common errors
Could not find adapter type postgres→ installdbt-postgres(or your adapter).Credentials in profile ... could not be parsed→ useenv_var()and provide the env in CI.Compilation Error ... depends on a package→ rundbt depsbefore build.
Key takeaways
- Install both dbt-core and the warehouse adapter.
- Read warehouse credentials from env via
env_var()in profiles.yml. - Run
dbt depsbeforedbt build.
Related guides
How to Install psycopg2 in CI Without libpq ErrorsInstall psycopg2 in CI: use psycopg2-binary or add libpq-dev for a source build, connect to a Postgres servic…
How to Execute Notebooks with nbconvert in CIRun and convert Jupyter notebooks in CI with nbconvert: install a kernel, execute headlessly, handle Pandoc/T…
How to Run Great Expectations in CIRun Great Expectations in CI: install the right datasource extras, point at a test database, run checkpoints…
Self-Healing CI: Auto-Retrying Transient Registry and 5xx FailuresRegistry timeouts, 429s, and 5xx errors are transient by definition. See why they happen and how self-healing…