Dictionary of Applied Machine Learning: 107 machine-learning terms, each with a definition, a figure, citations and a Python demo · All terms · Demos · Start here

data leakage

Updated on 2026-10-08

Readiness 97/100

▶ Run the Python demo Open in Colab Typeset PDF LaTeX source Cite this entry

See also training set validation set test set generalization

Data leakage arises when a machine learning (ML) method uses features that are available during training but not at prediction time. For example, in weather forecasting each data point covers a time interval, and one feature is the temperature one hour ahead of the forecast time. On historic data this later measurement is available, but when a forecast is actually computed it is not, so the feature leaks information. A leaked feature makes the loss on the training set and test set look near zero, yet the learned hypothesis fails after deployment, because the feature cannot be reproduced at prediction time. The term is also used, in a security sense, for the unintentional release of sensitive information.

Figure produced by the Python demo of the entry data leakage
Figure written by the entry's Python demo dataleakage.py, which recomputes what the entry states: the script block by block · open in Colab

Definition

B-deployConsider a machine learning (ML) method trained on an hourly temperature sequence, where each data point is a sliding window of three consecutive hours (Fig. 1). The label is the temperature of the middle hour, and one of the features is the measurement one hour later. On historic data this later measurement is available, so the loss on the training set and test set looks near zero; but in real time the middle hour must be predicted before the next hour is measured, so the learned hypothesis is useless. This is data leakage: a feature used during training or evaluation is unavailable when a prediction must be delivered.

Figure 1 of the entry dataleakage
Figure 1: Hourly temperature measurements. Each data point is a sliding window of three hours (box); the label is the middle hour's temperature. Using the measurement one hour later as a feature leaks information: at prediction time (dashed) that hour has not been measured yet, so the feature is available for historic training data only
More generally, a leaked feature is one whose value cannot be reproduced at prediction time, whether it is measured only later (as above) or computed from the label. Data leakage is formalized by Kaufman et al. (2012), who also give methods to detect and avoid it. Kapoor and Narayanan (2023) survey it as a cause of the reproducibility crisis in ML-based science. Test set contamination, the presence of test set data points in the training set, is a related form of data leakage.

The term data leakage is also used, in an unrelated security sense, for the unintentional release of sensitive information (data exfiltration from a trained system), a meaning distinct from the methodological one defined above.

See also: test set contamination, training set, validation set, test set, generalization, risk.

References

  1. Kaufman et al. (2012). Leakage in Data Mining: Formulation, Detection, and Avoidance. ACM Transactions on Knowledge Discovery from Data. doi.org/10.1145/2382577.2382579
  2. Kapoor and Narayanan (2023). Leakage and the Reproducibility Crisis in Machine-Learning-Based Science. Patterns. doi.org/10.1016/j.patter.2023.100804

Cite this entry

@misc{dictml_dataleakage,
  author = {Jung, Alexander},
  editor = {Olioumtsevits, Konstantina and Schnoor, Ekkehard},
  title = {data leakage},
  howpublished = {Dictionary of Applied Machine Learning (course edition)},
  year = {2026},
  doi = {10.5281/zenodo.21569296},
  note = {ISBN 978-952-64-3013-3, CC BY 4.0, retrieved 2026-10-27},
  url = {https://dictionaryofml.org/terms/dataleakage.html}
}