Dictionary of Applied Machine Learning

random experiment

Updated on 2026-10-12

Typeset PDF Cite this entry

See also random variable probability distribution independent and identically distributed

A random experiment is a physical or computational process that produces an outcome $\outcome$ from a known set $\samplespace$ of possibilities, the sample space, without the outcome being known in advance. Probability theory represents the process by a probability space $\probspace$, where the $\sigma$-algebra $\sigmaalgebra$ collects the events and the probability measure $\prob{\cdot}$ assigns each event a probability; this representation is a probabilistic model of the process, not a property of it. A measurement of the outcome is represented by a random variable (RV), a measurable function $\feature : \samplespace \to \featurespace$, and the measured value $\feature(\outcome)$ is a realization. Repeating the experiment $\samplesize$ times is itself a random experiment on the product sample space, and the measurements of the runs are modeled as independent and identically distributed (i.i.d.) RVs. In machine learning (ML), the collection of a training set, the draw of a subset of it by stochastic gradient descent (SGD), and the noise added for differential privacy (DP) are random experiments.

Definition

Rolling a die, reading the maximum temperature of a day at the weather station Krems an der Donau, or drawing a handful of data points from a training set for one iteration of stochastic gradient descent (SGD): each of these is a random experiment. Its outcome is not known before the process is carried out, while the set of possible outcomes is. Formally, a random experiment is a physical or computational process that produces an outcome $\outcome$ from a set $\samplespace$ of possibilities, the sample space of the experiment (Bertsekas and Tsitsiklis, 2008, Sect. 1.2; Billingsley, 1986). Its defining characteristic is that the outcome is uncertain, and a key conceptual property is that it can be repeated under identical conditions.

Probability theory does not study the process itself but a mathematical representation of it, a probability space $\probspace$ (Bertsekas and Tsitsiklis, 2008, Sect. 1.2; Billingsley, 1986). The sample space $\samplespace$ collects the possible outcomes. The $\sigma$-algebra $\sigmaalgebra$ collects the events, those subsets $\genericevent \subseteq \samplespace$ to which a probability is assigned. The probability measure $\prob{\cdot}$ assigns to each event $\genericevent \in \sigmaalgebra$ a number $\prob{\genericevent} \in [0,1]$ such that $\prob{\samplespace} = 1$ and the probability of a countable union of pairwise disjoint events is the sum of their probabilities. Choosing $\samplespace$, $\sigmaalgebra$ and $\prob{\cdot}$ is a modeling step: the probability measure is a property of the probabilistic model, not of the process. For a die, the sample space is $\{1, \ldots, 6\}$ and the uniform probability measure is the model of a fair die; whether a given die is fair is a question about the process, answered by the relative frequencies of its outcomes in many rolls.

What is observed of an outcome is a measurement, a function $\feature : \samplespace \to \featurespace$ of the outcome. Such a measurable function is a random variable (RV), and the value $\feature(\outcome)$ it takes for the actual outcome $\outcome$ is a realization of it (Billingsley, 1986; Gray, 2009). The temperature read at 13:00 is an RV on the weather of a day, and the number read on one day is a realization. The probability measure on $\samplespace$ induces the probability distribution of the RV, $\probdist^{(\feature)}(\mathcal{B}) = \prob{\feature \in \mathcal{B}}$ for every measurable set $\mathcal{B} \subseteq \featurespace$. Machine learning (ML) methods work with the RVs and their probability distributions; the sample space behind them rarely needs to be written down. Fig. 1 depicts the two layers, the process and its mathematical representation.

Figure 1 of the entry randomexperiment
Figure 1: A random experiment and its mathematical representation. The process produces an outcome $\outcome$, of which a measurement $\feature(\outcome)$ is observed. Probability theory represents the process by a probability space $\probspace$ and the measurement by an RV $\feature$, a measurable function on $\samplespace$ whose probability distribution $\probdist^{(\feature)}$ is induced by $\prob{\cdot}$
Repeating a random experiment $\samplesize$ times defines a new random experiment whose outcomes are sequences $(\outcome^{(1)}, \ldots, \outcome^{(\samplesize)})$ of outcomes of the original one (Fig. 2). Its sample space is the product $\samplespace \times \cdots \times \samplespace$, and repetition under identical conditions, with no run influencing another, is represented by the product probability measure: the measurements $\feature^{(1)}, \ldots, \feature^{(\samplesize)}$ of the runs are then independent and identically distributed (i.i.d.) RVs (Bertsekas and Tsitsiklis, 2008, Sect. 1.5). While the outcome of a single run is uncertain, the long-run behavior of repeated runs becomes predictable: the law of large numbers and the central limit theorem (CLT) make this precise.
Figure 2 of the entry randomexperiment
Figure 2: Repeating a random experiment $\samplesize$ times yields another random experiment whose outcomes are sequences $(\outcome^{(1)}, \,\outcome^{(2)}, \,\dots, \,\outcome^{(\samplesize)}) \in \samplespace\times\ldots\times \samplespace$. One random experiment arising in many ML applications is the gathering of a training set $\datapoint^{(1)},\,\ldots,\,\datapoint^{(\samplesize)}$, whose data points are RVs on the repeated runs
Random experiments arise in ML in three places:

See also: outcome, sample space, event, random variable, realization, probability, probability space, probability measure, probability distribution, probabilistic model, independent and identically distributed, law of large numbers, central limit theorem.

References

  1. Bertsekas and Tsitsiklis (2008). Introduction to Probability. Athena Scientific.
  2. Billingsley (1986). Probability and Measure. Wiley.
  3. Gray (2009). Probability, Random Processes, and Ergodic Properties. Springer Science+Business Media. doi.org/10.1007/978-1-4419-1090-5

Cite this entry

@misc{dictml_randomexperiment,
  author = {Jung, Alexander},
  editor = {Olioumtsevits, Konstantina and Schnoor, Ekkehard},
  title = {random experiment},
  howpublished = {Dictionary of Applied Machine Learning (course edition)},
  year = {2026},
  doi = {10.5281/zenodo.21569296},
  note = {ISBN 978-952-64-3013-3, CC BY 4.0, retrieved 2026-10-12},
  url = {https://dictionaryofml.org/terms/randomexperiment.html}
}