What Is Data Drift? Shifting Input Distributions Explained

Data drift is a change over time in the statistical distribution of the input data a model sees, which can quietly degrade its predictions.

Data drift is one of the main reasons models get worse without anyone touching them. The features feeding the model shift away from what it learned on - new user segments, seasonal effects, a changed upstream source - and the model, trained on the old distribution, starts to miss.

What data drift is

Data drift (also called feature or covariate drift) is a change in the distribution of a model input features between training and production. The relationship between inputs and the target may still hold, but the inputs themselves now look different from the training data.

Data drift versus concept drift

Data drift is about the inputs changing distribution. Concept drift is about the input-to-output relationship changing. A model can suffer from one or both. Distinguishing them matters: data drift may be handled by retraining on fresh data, while concept drift may need new features.

How it is detected

Drift is detected by comparing recent production feature distributions against the training baseline using statistical tests (such as population stability index or Kolmogorov-Smirnov) or distance metrics. A significant divergence on important features flags potential drift.

Data drift checks in CI/CD

A scheduled CI job computes drift metrics against the baseline and can alert or trigger retraining when thresholds are crossed.

A data-drift check in CI

steps:
  - run: python drift_check.py --baseline train_ref.parquet \
        --current prod_recent.parquet --metric psi --threshold 0.2

Latchkey note

Drift checks pull large recent-data windows on a schedule. On Latchkey, caching the training baseline and dependencies keeps these recurring jobs cheap, and auto-retry covers transient reads while fetching the comparison window from the warehouse or lake.

Key takeaways

Data drift is a shift in the distribution of a model input features between training and production.
It differs from concept drift, which is a change in the input-to-output relationship.
Statistical tests against a training baseline detect drift and can trigger alerts or retraining in CI.