Dictionary of Applied Machine Learning
Updated on 2026-09-18
Typeset PDF version — the authoritative form of this entry
The independent and identically distributed assumption (i.i.d. assumption) is the probabilistic model that represents the data points of a dataset as realizations of independent and identically distributed (i.i.d.) random variables (RVs): each data point is drawn from the same probability distribution, independently of the others. For a fixed hypothesis, the average loss on a training set then has expectation equal to the risk, the expected loss on a new data point, which is what justifies empirical risk minimization (ERM) and the use of a validation set to estimate the loss outside the training set. The assumption fails under distribution shift, when the data points met after training follow a different probability distribution, and for data points recorded over time whose consecutive values are dependent.
The independent and identically distributed (i.i.d.) assumption is a widely used probabilistic model for the generation of data points. In particular, data points are represented as i.i.d. random variables (RVs).
See also: independent and identically distributed, probabilistic model, data point, random variable.
@misc{dictml_iidasspt,
author = {Jung, Alexander and Olioumtsevits, Konstantina and Schnoor, Ekkehard},
title = {independent and identically distributed assumption},
howpublished = {Dictionary of Applied Machine Learning (course edition)},
year = {2026},
doi = {10.5281/zenodo.21569296},
note = {ISBN 978-952-64-3013-3, CC BY 4.0, retrieved 2026-09-19},
url = {https://dictionaryofml.org/terms/iidasspt.html}
}