How to Validate Airflow DAGs With an Import Test in CI
Loading every DAG through DagBag surfaces import errors that only appear when the scheduler parses the file.
Run a pytest that builds a DagBag from your dags folder and asserts import_errors is empty, catching parse failures before deploy.
Steps
- Install
apache-airflowmatching your production version. - Point
AIRFLOW__CORE__DAGS_FOLDERat your dags directory. - Assert
DagBag().import_errorsis empty in a test. - Run the test on every pull request.
Test
tests/test_dag_validation.py
from airflow.models import DagBag
def test_no_import_errors():
dag_bag = DagBag(dag_folder="dags", include_examples=False)
assert dag_bag.import_errors == {}, dag_bag.import_errors
def test_dags_have_owner_and_tags():
dag_bag = DagBag(dag_folder="dags", include_examples=False)
for dag_id, dag in dag_bag.dags.items():
assert dag.tags, f"{dag_id} is missing tags"Workflow
.github/workflows/ci.yml
on: [pull_request]
jobs:
dags:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: '3.11'
- run: pip install "apache-airflow==2.9.3" pytest
- run: pytest tests/test_dag_validation.py -qGotchas
- Pin the Airflow version to match production so provider imports resolve the same way.
- A slow DAG top-level (network calls at parse time) will slow every scheduler loop; catch it here.
Related guides
How to Check Airflow DAG Parse Time in CIFail CI when an Airflow DAG file takes too long to parse using airflow dags list-import-errors and a timed pa…
How to Deploy Airflow DAGs to MWAA From CIDeploy validated Airflow DAGs to Amazon MWAA from CI by syncing the dags folder to the environment S3 bucket…