How to Pull DVC-Tracked Data in GitHub Actions
dvc pull restores the exact data and model files pinned by the .dvc metadata committed to Git, using a configured remote and secret credentials.
Install DVC, configure the remote credentials from a secret, then run dvc pull to materialize the data version referenced by the current commit before training or testing.
Steps
- Install DVC with the remote extra (e.g.
dvc[s3]). - Set remote credentials from repository secrets.
- Run
dvc pullto fetch the pinned data.
Workflow
.github/workflows/ci.yml
jobs:
data:
runs-on: ubuntu-latest
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
steps:
- uses: actions/checkout@v4
- run: pip install "dvc[s3]"
- run: dvc pull -j 4Gotchas
- Commit the
.dvcfiles anddvc.lockso CI pulls the exact version you tested locally. - Use
-jto parallelize the pull for many small files.
Related guides
How to Cache Model Weights and Datasets in GitHub ActionsCache large model weights and datasets between GitHub Actions runs with actions/cache keyed on a content hash…
How to Validate Data Before Training in GitHub ActionsRun schema and quality checks on a dataset in GitHub Actions before training, failing the run on nulls, out-o…