What Is DVC? Data Version Control Explained

DVC (Data Version Control) is an open-source tool that versions large datasets and models alongside Git and defines reproducible data pipelines.

DVC is one of the most popular tools for bringing version control to machine learning. It works with Git rather than replacing it: Git tracks your code and small pointer files, while DVC stores the large data and models in remote object storage.

What DVC is

DVC is a command-line tool, used like Git, that versions data and models too big to live in a Git repository. When you add a dataset with "dvc add", DVC stores the data in a cache and remote, and commits a small .dvc pointer file to Git so the version travels with your code.

Pipelines and reproducibility

Beyond versioning, DVC can define a pipeline in a dvc.yaml file with stages, dependencies, and outputs. Running "dvc repro" re-executes only the stages whose inputs changed, much like a build system, which makes ML pipelines reproducible and incremental.

Remotes and storage

DVC pushes data to a remote - S3, GCS, Azure Blob, or an SSH server. Teammates and CI jobs run "dvc pull" to fetch the exact data versions a commit references, keeping the heavy bytes out of Git while preserving reproducibility.

DVC in CI

A CI job checks out the repo, pulls the pinned data, and reproduces the pipeline, so every run trains on exactly the intended data.

Using DVC in a CI job

steps:
  - uses: actions/checkout@v4
  - run: pip install dvc[s3]
  - run: dvc pull          # fetch pinned data + models
  - run: dvc repro         # rerun changed pipeline stages

Latchkey note

DVC pulls can move gigabytes from object storage on every run. On Latchkey, caching the DVC cache directory between runs avoids re-downloading unchanged data, and auto-retry on flaky S3 or GCS reads keeps a long training pipeline from failing on a single transient blip.

Key takeaways

DVC versions large datasets and models alongside Git, keeping the heavy bytes in remote object storage.
Its dvc.yaml pipelines and "dvc repro" make ML workflows reproducible and incremental.
In CI, "dvc pull" plus a cached DVC directory restores the exact data a commit references, fast.