What Is Model Drift? Model Degradation Over Time Explained

Model drift is the gradual decline in a deployed model accuracy as the real-world data it sees diverges from the data it was trained on.

A model is a snapshot of the world at training time. The world keeps moving - customer behavior shifts, prices change, new patterns emerge - and the model slowly falls out of date. This decay is model drift, and detecting it is essential to keeping ML systems trustworthy.

What model drift is

Model drift is the degradation of a model predictive performance over time as production conditions change. The model has not changed; the world it predicts has. Left unaddressed, a once-accurate model quietly makes worse and worse predictions.

Types of drift

Two common forms are data drift, where the distribution of the input features changes, and concept drift, where the relationship between inputs and the target changes. A model can degrade from either, and they often require different responses.

How drift is detected

Drift is detected by monitoring: comparing the statistical distribution of recent inputs and predictions against the training baseline, and tracking live accuracy where ground truth becomes available. Significant divergence raises an alert or triggers retraining.

Drift and CI/CD

Drift detection ties monitoring back into the pipeline. A scheduled CI job can compute drift metrics and, if they cross a threshold, trigger a continuous-training run to refresh the model.

A drift-check job that can trigger retraining

steps:
  - run: python detect_drift.py --baseline ref.parquet
  - run: python detect_drift.py --threshold 0.2 || gh workflow run retrain.yml

Latchkey note

Drift checks read large windows of production data on a schedule. On Latchkey, caching the training baseline and dependencies keeps these recurring jobs cheap, and auto-retry covers transient warehouse or object-store reads while pulling the comparison data.

Key takeaways

Model drift is the decline in a deployed model accuracy as production data diverges from training data.
Data drift changes the input distribution; concept drift changes the input-to-target relationship.
Monitoring detects drift and can trigger a continuous-training run to refresh the model.