Dictionary of Applied Machine Learning: 147 machine-learning terms, each with a definition, a figure, citations and a Python demo · All terms · Demos · Start here

training

Updated on 2026-11-20

Readiness 100/100

▶ Run the Python demo Open in Colab Typeset PDF LaTeX source Markdown Cite this entry

See also supervised learning unsupervised learning reinforcement learning empirical risk minimization gradient descent stochastic gradient descent online learning expectation–maximization Gaussian mixture model $k$-means model loss

Training is the process of choosing a useful hypothesis out of a hypothesis space, or model, by solving an optimization problem whose objective function a training method builds from the data points it reads. In supervised learning, the objective function is the average loss over data points whose label values are known, such as the average squared error loss. In unsupervised learning, it measures structure instead, such as the clustering error incurred by a choice of cluster centroids. In reinforcement learning (RL), it is the expected cumulative reward collected by a policy. Training methods differ also in how they access data points: offline methods require the training set up front, while online methods update the hypothesis whenever new data points arrive. Many training methods iterate an operator on the model parameters of a parametric model: a gradient step where the objective function is differentiable, and one sweep of alternating optimization in the $k$-means and expectation–maximization (EM) algorithms. An offline method applies the same operator at every step and stops at a fixed point of it, while an online method applies an operator that the arriving data point changes. These operators make the objective function small on the data points the method is given, and controlling the loss on data points outside the training set is a challenge that no choice of optimization method removes.

Figure produced by the Python demo of the entry training
Figure written by the entry's Python demo training.py, which recomputes what the entry states: the script block by block · open in Colab

Definition

A shop forecasts daily bread demand from the weather with a linear model. It trains that linear model by gradient descent (GD) on a year of past records. To let the forecaster follow a changing clientele instead, it trains the same linear model by online GD on each new day's sales: the same model and the same objective function, with a different form of data access. The shop trains in both cases. Training is the process by which a machine learning (ML) method picks one hypothesis out of a hypothesis space. The hypothesis space $\mathcal{H}$, or model, is the set of candidate maps the method may choose from. Training selects the single hypothesis $\widehat{h}$ that the method then uses for inference: the training phase learns $\widehat{h}$, and the inference phase applies $\widehat{h}$, unchanged, to newly arriving data points to deliver predictions. What makes one candidate better than another is an objective function, so training amounts to solving an optimization problem.

Training methods differ in two respects. The first is the objective function. In supervised learning, each data point carries a label, and the objective function is the average loss of a hypothesis over the training set, its empirical risk, such as the average squared error loss in linear regression. In unsupervised learning, no label is given, so the objective function measures a structural quantity instead, such as the clustering error incurred by a choice of cluster centroids in hard clustering. In reinforcement learning (RL), there is no fixed training set at all: the objective function is the expected cumulative reward $G$ collected by a policy that interacts with an environment.

The second respect is the form of data access, offline or online. A training method is an algorithm $\mathcal{A}$ that reads in a dataset $\mathcal{D}$ of data points, constructs the objective function from them together with the chosen model and loss, and delivers the hypothesis making that objective function small, $\widehat{h} = \mathcal{A}(\mathcal{D})$. An offline algorithm reads in all data points at once, computes, and delivers $\widehat{h}$. An online algorithm reads in new data points as they arrive, updating the current hypothesis at each time instant $t = 1, 2, \ldots$ instead of waiting for a complete training set. The hypotheses it produces form a Markov chain, since each one depends on the dataset only through its predecessor and the data point that just arrived (Fig. 1).

Figure 1 of the entry training
Figure 1: Two forms of data access. In (a) an offline algorithm $\algomap$ reads in the entire dataset $\dataset$ and delivers a learned hypothesis $\learnthypothesis$. In (b) an online algorithm holds a current hypothesis $\learnthypothesis^{(\timeidx)}$, which the data point $\datapoint^{(\timeidx)}$ arriving at time instant $\timeidx$ moves to $\learnthypothesis^{(\timeidx+1)}$. The hypotheses in (b) therefore form a Markov chain: the next one depends on the dataset only through the current one and the arriving data point
Inference and online training both act on a newly arriving data point, and that data point plays a different role in each. Inference applies a hypothesis that training has already delivered and leaves it as it is, so the arriving data point supplies only the features to predict from. Online training keeps changing the hypothesis, so the arriving data point supplies the loss that decides the next hypothesis. An online method does both in each round: it commits to a hypothesis and predicts with it, the loss of that hypothesis is then revealed, and the next hypothesis is chosen from the losses revealed so far (Hazan, 2022, Sect. 1.1).

B-sgdFor a parametric model, each hypothesis $h^{({\mathbf w})}$ is fixed by a choice of model parameters ${\mathbf w}$, so training searches over ${\mathbf w}$. Many training methods amount to iterating an operator on those model parameters, \[ {\mathbf w}^{(t+1)} = T({\mathbf w}^{(t)}) \text{, for } t = 0,1,\ldots \text{,} \] and the two forms of data access appear here as two forms of the operator. An offline method builds its objective function once, from the whole dataset, so $T$ is fixed: the same map is applied at every step, and training stops at, or near, a fixed point of $T$, where a further iteration leaves the model parameters unchanged. An online method has no fixed objective function. Its operator reads in the arriving data point, \[ {\mathbf w}^{(t+1)} = T\big({\mathbf w}^{(t)}, {\mathbf z}^{(t)}\big) \text{,} \] so the map changes from step to step, and in general there is no single fixed point to stop at. The model parameters keep changing for as long as new data points arrive. The learned hypothesis $\widehat{h}$ is the one that the final model parameters define.

B-onlineWhat $T$ does is decided by the shape of the objective function. When the objective function is differentiable in all model parameters at once, $T$ is a gradient step, $T({\mathbf w}) = {\mathbf w} - \eta \nabla f({\mathbf w})$ with step size $\eta$ (Boyd and Vandenberghe, 2004, Sect. 9.3). The form of data access then decides which objective function the gradient is taken of, so a single operator yields three different training methods. Offline empirical risk minimization (ERM) takes the gradient of the average loss $f({\mathbf w})$ over the whole training set. When that training set is too large to pass through at every step, stochastic gradient descent (SGD) replaces $\nabla f({\mathbf w})$ by the gradient of the average loss over a few data points (Bottou, 2010). Its operator is drawn at random at each step, from a distribution that the training set fixes, so it is fixed in expectation rather than fixed outright. In online learning, there is no training set at all, and online GD takes one gradient step on the loss of the arriving data point alone (Zinkevich, 2003).

B-kmeansSome objective functions arising in training methods can be minimized easily once a subset of the model parameters is held fixed. Then $T$ is one sweep of alternating optimization, a minimization over each subset in turn (Beck, 2017, Ch. 14). The $k$-means algorithm alternates between assigning each data point to the nearest cluster and recomputing the cluster means (Lloyd, 1982). The expectation–maximization (EM) algorithm is the probabilistic instance: training a Gaussian mixture model (GMM), it alternates between the responsibilities of the mixture components for each data point and the parameters of those components (Dempster et al., 1977). What separates EM from a plain alternation is that each step minimizes a surrogate, a bound on the objective function rather than the objective function itself, which is why the likelihood function cannot decrease (Bishop, 2006, Sect. 9.4). Both are offline. The $k$-means algorithm reaches a fixed point of its operator after finitely many steps, since every step lowers the objective function and there are finitely many assignments (Bishop, 2006, Sect. 9.1); for EM, under a mild continuity condition, the likelihood function converges and every limit point of the iterates is a stationary point of it (Wu, 1983, Thm. 2). Neither is guaranteed to reach the smallest loss available in $\mathcal{H}$: $k$-means can stop at a local minimum of its objective function (Bishop, 2006, Sect. 9.1), and a stationary point of the likelihood function need not maximize it.

A gradient step and a sweep of alternating optimization both make the objective function small on the data points the training method is given, and that is all either of them does. Controlling the discrepancy between the loss on the training set and the loss on data points outside it is one of the main challenges in ML, and no choice of optimization method removes it (Goodfellow et al., 2016).

Synonyms: model fitting, fitting.

See also: optimization problem, inference, supervised learning, unsupervised learning, reinforcement learning, empirical risk minimization, gradient descent, stochastic gradient descent, online learning, expectation–maximization, Gaussian mixture model, $k$-means, fixed point, model, model parameter, loss.

References

  1. Hazan (2022). Introduction to Online Convex Optimization. MIT Press. sites.google.com/view/intro-oco/
  2. Boyd and Vandenberghe (2004). Convex Optimization. Cambridge Univ. Press. doi.org/10.1017/CBO9780511804441
  3. Bottou (2010). Large-scale machine learning with stochastic gradient descent. Proceedings of COMPSTAT'2010. doi.org/10.1007/978-3-7908-2604-3_16
  4. Zinkevich (2003). Online Convex Programming and Generalized Infinitesimal Gradient Ascent. Proc. 20th Int. Conf. Mach. Learn..
  5. Beck (2017). First-Order Methods in Optimization. SIAM-Society for Industrial and Applied Mathematics.
  6. Lloyd (1982). Least squares quantization in PCM. IEEE Trans. Inf. Theory. doi.org/10.1109/TIT.1982.1056489
  7. Dempster et al. (1977). Maximum likelihood from incomplete data via the EM algorithm. J. Roy. Statist. Soc.: Ser. B (Methodological). doi.org/10.1111/j.2517-6161.1977.tb01600.x
  8. Bishop (2006). Pattern Recognition and Machine Learning. Springer Science+Business Media. doi.org/10.1007/978-0-387-45528-0
  9. Wu (1983). On the convergence properties of the EM algorithm. Ann. Statist.. doi.org/10.1214/aos/1176346060
  10. Goodfellow et al. (2016). Deep Learning. MIT Press.

Cite this entry

@misc{dictml_training,
  author = {Jung, Alexander},
  editor = {Olioumtsevits, Konstantina and Schnoor, Ekkehard},
  title = {training},
  howpublished = {Dictionary of Applied Machine Learning (course edition)},
  year = {2026},
  doi = {10.5281/zenodo.21569296},
  note = {ISBN 978-952-64-3013-3, CC BY 4.0, retrieved 2026-11-20},
  url = {https://dictionaryofml.org/terms/training.html}
}