How to Add a Data Diff Check in a Pull Request
A data diff compares the PR-built model to production so reviewers see exactly which rows and columns changed.
Build the changed models in CI, then run recce (or data-diff) to compare them against the production relations and post a summary.
Steps
- Build changed models against the CI schema.
- Run
recce runwith the production manifest as the base. - Attach the recce summary as an artifact or PR comment.
Workflow
.github/workflows/ci.yml
steps:
- uses: actions/checkout@v4
- run: pip install dbt-snowflake==1.8.0 recce
- run: aws s3 cp s3://dbt-artifacts/manifest.json target-base/manifest.json
- run: dbt build --target ci --select state:modified+ --defer --state target-base
- name: Run data diff
run: recce run --target-base-path target-base
- uses: actions/upload-artifact@v4
with:
name: recce-report
path: recce_state.jsonGotchas
- Diff a key subset of columns for wide tables so the comparison stays cheap.
- A large diff is a signal to review, not always a failure; decide which diffs should block.
Related guides
How to Test Incremental dbt Models in CIVerify dbt incremental models in CI by running a full refresh then an incremental run over new data, assertin…
How to Control the Cost of Running Data Tests in CICut the warehouse cost of data tests in CI by limiting rows with a sample, using slim CI selection, and auto-…