DVC — Version Control for Data and Models

What DVC is

DVC — Data Version Control — extends Git’s model to files Git cannot handle. dvc add data/raw/ computes a checksum, writes a small pointer file that Git tracks, and moves the actual bytes to storage you configure: S3, a university NAS, an SSH server, Google Drive. Checking out an old commit and running dvc pull restores exactly the data that existed at that commit.

The problem it solves is specific and widespread. Git is excellent for text and terrible for large binaries — it stores every version in full, repositories bloat, and hosts impose size limits well below any real dataset. So data ends up outside version control, in a folder called data_v2_final, with no reliable link between a result and the data that produced it. DVC restores that link without putting gigabytes in Git: the repository holds the reference, and the storage holds the bytes.

Why researchers use it

A commit identifies its data — the question “which dataset produced Figure 3” becomes a checkout rather than an archaeology exercise.
Your bytes stay where you choose — institutional storage, a lab NAS, or any cloud. DVC is a pointer system, not a hosting service, so governance stays yours.
Git habits transfer — dvc add, dvc push, dvc pull mirror the commands you already know.
Pipelines with caching — dvc repro reruns stages whose inputs changed, which for ML experiments is the difference between rerunning a step and rerunning a week.
Experiment metrics in Git — small metrics files versioned alongside code, so comparing runs is a diff.

Where it fits in a research workflow

DVC sits between GitHub and your storage, filling the gap Git leaves. Code, environment definitions, and DVC pointer files go in the repository; the data goes wherever your institution allows data to live.

It is complementary to the pipeline tools rather than an alternative to them, though the overlap causes genuine confusion. Snakemake and Nextflow orchestrate computation across many steps and scale to clusters; DVC’s pipeline features are lighter and are really about caching in a versioning context. Running both means two systems with opinions about what is out of date, which is worth avoiding — pick DVC for versioning and Snakemake for orchestration, and do not use DVC’s pipelines if you already have one. At publication, the versioned data is deposited to Zenodo for the citable record. For ML specifically, Weights & Biases covers experiment metrics more richly, and the two are commonly used together.

Getting started

An afternoon, and configure the remote before you commit anything.

In an existing Git repository, run dvc init, then dvc add data/raw/. Commit the generated .dvc pointer file — not the data.
Configure a remote: dvc remote add -d store s3://bucket/path or an SSH path to institutional storage, then dvc push. Without a remote, DVC is tracking data nobody else can retrieve.
Have a collaborator run git pull then dvc pull and confirm they get the identical dataset. This is the test that proves the setup works.
The step people skip: write down the remote configuration and access requirements in the README. A repository whose data lives on storage nobody else can reach is a repository with a broken promise, and the person who discovers that is usually a reviewer.

DVC vs the alternatives

Alternative	Does it better	Pick it if
Git LFS	Simpler setup, integrated with GitHub	Files are moderately large and hosting quota is acceptable
Zenodo	Permanent citable archival with a DOI	The data is finished and being published
Snakemake	Orchestrating computation across many steps and machines	Your problem is the pipeline, not the versioning
A checksum in the README	Nothing, but it costs nothing	The dataset is downloaded once and never changes

That last row is the honest comparison. If your data never changes, a documented checksum answers the same question DVC does, for free.

Cost, licensing, and your data

Free and open source under Apache 2.0, from Iterative, which sells commercial tooling around it. There is no hosting cost from DVC itself because it does not host anything — you supply the storage, and that is the design.

That storage-agnosticism is the governance advantage. Data tracked with DVC can live on institutional infrastructure, inside your jurisdiction, under your existing data agreements, while the versioning happens in a public repository that contains only checksums and paths. For a project with participant data and a public analysis repository, that separation is exactly right — the code is open and the data is controlled, with a verifiable link between them. Check that the pointer files themselves do not leak informative paths, which is a small thing that occasionally matters.

The honest review

Strengths. DVC answers the question reviewers actually ask — which data produced this result — with a command rather than a story. Because it separates the reference from the bytes, it does this without forcing your data into a hosting service, which is what makes it usable for projects where the data cannot leave institutional storage. For machine learning, where datasets and models evolve alongside code across hundreds of experiments, that link is the difference between a reproducible result and a plausible one.

Limitations. It is another layer of mental overhead on top of Git, with its own state, its own cache, and its own failure modes — and the failures are confusing, because a dvc pull that half-succeeds leaves you with a repository that looks fine and is not. Everyone on the team must adopt it or the pointers rot silently. The pipeline features overlap awkwardly with Snakemake and Nextflow, and running two systems that both track staleness produces genuine confusion. And for static data it is pure bureaucracy: a dataset downloaded once and never modified gains nothing that a checksum does not provide.

Verdict. Adopt it when your data changes alongside your code — machine learning, iterative preprocessing, anything where “version 3 of the cleaned dataset” is a real object. Skip it for static data, small files that fit in Git, and solo projects with one dataset. The condition that flips the answer is whether the data has versions worth distinguishing.

When NOT to use this Do not adopt DVC alone on a shared project. It works by convention: if one collaborator commits data directly, or pushes pointer files without pushing to the remote, the repository silently loses the link between code and data — and the failure surfaces months later when someone cannot reproduce a figure. Either the whole team adopts it and the remote is documented and reachable, or the project is better served by a checksum in the README and honest discipline.

Common questions

Is DVC free?

Yes — free and open source under Apache 2.0. It hosts nothing, so there are no storage fees from DVC; you pay for whatever storage you point it at, which may well be institutional and free to you.

DVC or Git LFS?

Git LFS is simpler and integrates directly with GitHub, but the data counts against your hosting quota and lives with the provider. DVC keeps bytes in storage you choose — including institutional infrastructure — which matters when data governance restricts where it can live.

Does DVC replace Snakemake?

No. DVC’s pipeline features are lightweight caching in a versioning context; Snakemake and Nextflow are full workflow engines that scale to clusters. Use DVC for data versioning and a workflow manager for orchestration, and avoid running both pipeline systems on the same project.