Dictionary of Applied Machine Learning

hypothesis

Typeset PDF version — the authoritative form of this entry

A hypothesis is a map $\hypothesis: \featurespace \rightarrow \labelspace$ from the feature space $\featurespace$ to the label space $\labelspace$. Given a data point with features $\featurevec$, the hypothesis returns a prediction $\hypothesis(\featurevec)$ for the label $\truelabel$. A machine learning (ML) method selects a single learned hypothesis $\learnthypothesis$ from a restricted hypothesis space $\hypospace \subseteq \labelspace^{\featurespace}$. This restriction reflects finite computational resources of a practical ML method. As a point in case, empirical risk minimization (ERM)-based methods must be able to search the entire hypothesis space to learn a hypothesis $\learnthypothesis$ that minimizes the empirical risk on a training set.

Definition

Predicting tomorrow's maximum daytime temperature from today's morning temperature is a typical machine learning (ML) task. A hypothesis is the map that performs this prediction. Formally, a hypothesis is a map (also called a function) $\hypothesis: \featurespace \rightarrow \labelspace$ from the feature space $\featurespace$ to the label space $\labelspace$ (Shalev-Shwartz and Ben-David, 2014, Sect. 2.1). Given a data point with features $\featurevec$, the hypothesis $\hypothesis$ returns an estimate $\predictedlabel = \hypothesis(\featurevec)$ of the true label $\truelabel$. Fig. 1 illustrates this for an audio-based application where the hypothesis predicts a Freddie-likeness score from an audio recording.

Figure 1 of the entry hypothesis
Figure 1: Hypothesis $\hypothesis: \featurespace \rightarrow \labelspace$ mapping the features $\featurevec \in \featurespace$ of a data point to a prediction $\hypothesis(\featurevec) \in \labelspace$ of the label. Here, the ML application https://freddiemeter.withyoutube.com/ uses the samples of an audio recording as features to predict how closely a person's singing resembles that of Freddie Mercury. The displayed value $\hypothesis(\featurevec)=0.82$ indicates an $82\%$ resemblance
An ML method aims to find (or learn) a hypothesis $\hypothesis$ such that $\truelabel \approx \hypothesis(\featurevec)$ for any data point with features $\featurevec$ and label $\truelabel$. Finite computational resources require restricting the search to a subset of \[ \labelspace^{\featurespace} = \big\{ \hypothesis: \featurespace \rightarrow \labelspace \big\} \text{,} \] the set of all functions from $\featurespace$ to $\labelspace$. This subset is the hypothesis space $\hypospace$ (also called the model) underlying the method. The output of an ML algorithm acting on a training set is a single learned hypothesis $\learnthypothesis \in \hypospace$ obtained via empirical risk minimization (ERM), i.e., the element of $\hypospace$ that minimizes the empirical risk on the training set.

Different hypothesis spaces admit different representations of their hypothesis maps. Polynomial regression (with a single scalar feature $\feature$) represents each $\hypothesis \in \hypospace$ as an algebraic expression $\hypothesis(\feature) = \sum_{j=0}^{\polydegree} \weight_{j} \feature^{j}$, with coefficients $\weight_{j}$ as the model parameters. A hypothesis space of computer programs represents $\hypothesis$ as executable source code, e.g., a Python function with a prescribed signature. A decision tree or an artificial neural network (ANN) represents $\hypothesis$ as a signal-flow chart whose nodes apply simple operations (e.g., comparisons, weighted sums, nonlinear activations) and whose edges define the order in which these operations are composed. These representations differ in which functions they can express and in how efficiently the ERM problem can be solved over the corresponding $\hypospace$. Fig. 2 shows the three representations side by side.

Figure 2 of the entry hypothesis
Figure 2: Three representations of a hypothesis map. (a) As an algebraic expression, here a degree-2 polynomial in a scalar feature $\feature$. (b) As source code, here a Python function. (c) As a signal-flow chart, here a small decision tree that routes a data point to a prediction of $3$, $1$, or $-1$ based on the threshold comparisons on the feature $\feature$
Synonyms: predictor.

See also: map, function, prediction, predictor, model, hypothesis space, feature vector, label, data point, empirical risk minimization, training, generalization.

References

  1. Shalev-Shwartz and Ben-David (2014). Understanding Machine Learning: From Theory to Algorithms. Cambridge Univ. Press.

Cite this entry

@misc{dictml_hypothesis,
  author = {Jung, Alexander},
  title = {hypothesis},
  howpublished = {Dictionary of Applied Machine Learning (course edition)},
  year = {2026},
  doi = {10.5281/zenodo.21569296},
  note = {ISBN 978-952-64-3013-3, CC BY 4.0, retrieved 2026-08-06},
  url = {https://dictionaryofml.org/terms/hypothesis.html}
}