Dictionary of Applied Machine Learning
Typeset PDF version — the authoritative form of this entry
Interpreting data points as independent and identically distributed (i.i.d.) realizations of a random variable (RV) is a basic device for the analysis of machine learning (ML) methods. The typical properties of such data points are governed by the probability distribution of this RV. The probability distribution of a binary RV is fully specified by a single probability. The probability distribution of a real-valued RV might be specified by a probability density function (pdf). In the most general case, a probability distribution is defined by a probability measure.
To analyze machine learning (ML) methods, it can be useful to interpret data points as independent and identically distributed (i.i.d.) realizations of a random variable (RV). The typical properties of such data points are then governed by the probability distribution of this RV. The probability distribution of a binary RV $\truelabel \in \{0,1\}$ is fully specified by the probabilities $\prob{\truelabel = 0}$ and $\prob{\truelabel=1}\!=\!1\!-\!\prob{\truelabel=0}$. The probability distribution of a real-valued RV $\feature \in \mathbb{R}$ might be specified by a probability density function (pdf) $p(\feature)$ such that $\prob{ \feature \in [a,b] } \approx p(a) |b-a|$. In the most general case, a probability distribution is defined by a probability measure (Billingsley, 1986; Gray, 2009).
See also: independent and identically distributed, realization, random variable, probability, probability density function.
@misc{dictml_probdist,
author = {Jung, Alexander},
title = {probability distribution},
howpublished = {Dictionary of Applied Machine Learning (course edition)},
year = {2026},
doi = {10.5281/zenodo.21569296},
note = {ISBN 978-952-64-3013-3, CC BY 4.0, retrieved 2026-08-06},
url = {https://dictionaryofml.org/terms/probdist.html}
}