# training *Entry of the [Dictionary of Applied Machine Learning](https://dictionaryofml.org) (course edition), by Alexander Jung, Aalto University. CC BY 4.0. Updated 2026-11-10. [HTML](https://dictionaryofml.org/terms/training.html) · [typeset PDF](https://dictionaryofml.org/terms/training.pdf) · [LaTeX](https://dictionaryofml.org/terms/training.tex) · [Python demo](https://dictionaryofml.org/terms/training.py) · [notebook](https://dictionaryofml.org/terms/training.ipynb)* Math is LaTeX with the book's macros expanded, so no notation package is needed to read it. ## Abstract Training is the process of choosing a useful hypothesis out of a hypothesis space, or model. Mathematically, training is an optimization problem with an objective function that a training method builds from the data points it reads. In supervised learning the objective function is the average loss over data points whose label values are known, such as the average squared error loss. In unsupervised learning it measures structure instead, such as the clustering error incurred by a choice of cluster centroids. In reinforcement learning it is the expected cumulative reward collected by a policy. Training methods differ also in how they access data points. Offline methods require the training set up front, while online methods learn a hypothesis sequentially, updating it whenever new data points arrive. ## Definition A shop forecasts daily bread demand from the weather with a linear model. It trains that linear model by gradient descent on a year of past records. To let the forecaster follow a changing clientele instead, it trains the same linear model by online gradient descent on each new day's sales: the same model and the same objective function, with a different form of data access. Training is the process by which a machine learning method picks one hypothesis out of a hypothesis space. The hypothesis space, or model, is the set of candidate maps the method may choose from. Training selects the single hypothesis $\widehat{h}$ that the method then uses for inference: the training phase learns $\widehat{h}$, the inference phase uses it to deliver predictions for newly arriving data points. What makes one candidate better than another is an objective function, so training amounts to solving an optimization problem. Training methods differ in two respects. The first is the objective function. In supervised learning each data point carries a label, and the objective function is the average loss of a hypothesis over the training set, its empirical risk, such as the average squared error loss in linear regression. In unsupervised learning no label is given, so the objective function measures a structural quantity instead, such as the clustering error incurred by a choice of cluster centroids in hard clustering. In reinforcement learning there is no fixed training set at all: the objective function is the expected cumulative reward $G$ collected by a policy that interacts with an environment. The second respect is the form of data access, offline or online. A training method is an algorithm $\mathcal{A}$ that reads in a dataset $\mathcal{D}$ of data points, constructs the objective function from them together with the chosen model and loss, and delivers the hypothesis making that objective function small, $\widehat{h} = \mathcal{A}(\mathcal{D})$. An offline algorithm reads in all data points at once, computes, and delivers $\widehat{h}$. An online algorithm reads in new data points as they arrive, updating the current hypothesis at each time instant $t = 1, 2, \ldots$ instead of waiting for a complete training set. The hypotheses it produces form a Markov chain, since each one depends on the dataset only through its predecessor and the data point that just arrived (Fig. 1). ![Figure 1 of the entry training](https://dictionaryofml.org/terms/fig/training-fig1.svg) *Figure 1: Two forms of data access. Left: an offline algorithm $\mathcal{A}$ reads in the entire dataset $\mathcal{D}$ and delivers a learned hypothesis $\widehat{h}$. Right: an online algorithm holds a current hypothesis $\widehat{h}^{(t)}$; the data point ${\mathbf z}^{(t)}$ arriving at time instant $t$ moves it to $\widehat{h}^{(t+1)}$, so the hypotheses form a Markov chain: the next one depends on the dataset only through the current one and the arriving data point* For a parametric model, each hypothesis $h^{({\mathbf w})}$ is fixed by a choice of model parameters ${\mathbf w}$, so training searches over ${\mathbf w}$. Many training methods amount to iterating an operator on those model parameters, $$ {\mathbf w}^{(t+1)} = T({\mathbf w}^{(t)}) \text{, for } t = 0,1,\ldots \text{,} $$ and the two forms of data access appear here as two forms of the operator. An offline method builds its objective function once, from the whole dataset, so $T$ is fixed: the same map is applied at every step, and training stops at, or near, a fixed point of $T$, where a further iteration leaves the model parameters unchanged. An online method has no fixed objective function. Its operator reads in the arriving data point, $$ {\mathbf w}^{(t+1)} = T\big({\mathbf w}^{(t)}, {\mathbf z}^{(t)}\big) \text{,} $$ so the map changes from step to step, and in general there is no single fixed point to stop at. The model parameters keep changing for as long as new data points arrive. The learned hypothesis $\widehat{h}$ is the one that the final model parameters define. What $T$ does is decided by the shape of the objective function. When the objective function is differentiable in all model parameters at once, $T$ is a gradient step, $T({\mathbf w}) = {\mathbf w} - \eta \nabla f({\mathbf w})$ with step size $\eta$ (Boyd and Vandenberghe, 2004, Sect. 9.3). The familiar methods are then one operator applied to three different objective functions, which is the second respect, data access, again. Offline empirical risk minimization takes the gradient of the average loss $f({\mathbf w})$ over the whole training set. When that training set is too large to pass through at every step, stochastic gradient descent replaces $\nabla f({\mathbf w})$ by the gradient of the average loss over a few data points (Bottou, 2010). Its operator is drawn at random at each step, from a distribution that the training set fixes, so it is fixed in expectation rather than fixed outright. In online learning there is no training set at all, and online gradient descent takes one gradient step on the loss of the arriving data point alone (Zinkevich, 2003). Some objective functions arising in training methods can be minimized easily once a subset of the model parameters is held fixed. Then $T$ is one sweep of alternating optimization, a minimization over each subset in turn (Beck, 2017, Ch. 14). The k-means algorithm alternates between assigning each data point to the nearest cluster and recomputing the cluster means (Lloyd, 1982). The expectation–maximization algorithm is the probabilistic instance: training a Gaussian mixture model, it alternates between the responsibilities of the mixture components for each data point and the parameters of those components (Dempster et al., 1977). What separates expectation–maximization from a plain alternation is that each step minimizes a surrogate, a bound on the objective function rather than the objective function itself, which is why the likelihood function cannot decrease (Bishop, 2006, Sect. 9.4). Both are offline. The k-means algorithm reaches a fixed point of its operator after finitely many steps, since every step lowers the objective function and there are finitely many assignments (Bishop, 2006, Sect. 9.1); for expectation–maximization, under a mild continuity condition, the likelihood function converges and every limit point of the iterates is a stationary point of it (Wu, 1983, Thm. 2). Neither is guaranteed to reach the smallest loss available in $\mathcal{H}$: $k$-means can stop at a local minimum of its objective function (Bishop, 2006, Sect. 9.1), and a stationary point of the likelihood function need not maximize it. Every operator above makes the objective function small on the data points it is given. One of the main challenges in machine learning is to control the discrepancy between the loss on the training set and the loss on data points outside it, and no choice of optimization method removes that challenge (Goodfellow et al., 2016). **Synonyms:** model fitting, fitting. **See also:** optimization problem, inference, [supervised learning](https://dictionaryofml.org/terms/supervisedlearning.html), [unsupervised learning](https://dictionaryofml.org/terms/unsupervisedlearning.html), [reinforcement learning](https://dictionaryofml.org/terms/reinforcementlearning.html), [empirical risk minimization](https://dictionaryofml.org/terms/erm.html), [gradient descent](https://dictionaryofml.org/terms/gd.html), [stochastic gradient descent](https://dictionaryofml.org/terms/stochGD.html), [online learning](https://dictionaryofml.org/terms/onlinelearning.html), [expectation–maximization](https://dictionaryofml.org/terms/em.html), [Gaussian mixture model](https://dictionaryofml.org/terms/gmm.html), [$k$-means](https://dictionaryofml.org/terms/kmeans.html), fixed point, [model](https://dictionaryofml.org/terms/model.html), model parameter, [loss](https://dictionaryofml.org/terms/loss.html). ## References - Boyd and Vandenberghe (2004). Convex Optimization. *Cambridge Univ. Press*. - Bottou (2010). Large-scale machine learning with stochastic gradient descent. *Proceedings of COMPSTAT'2010*. - Zinkevich (2003). Online Convex Programming and Generalized Infinitesimal Gradient Ascent. *Proc. 20th Int. Conf. Mach. Learn.*. - Beck (2017). First-Order Methods in Optimization. *SIAM-Society for Industrial and Applied Mathematics*. - Lloyd (1982). Least squares quantization in PCM. *IEEE Trans. Inf. Theory*. - Dempster et al. (1977). Maximum likelihood from incomplete data via the EM algorithm. *J. Roy. Statist. Soc.: Ser. B (Methodological)*. - Bishop (2006). Pattern Recognition and Machine Learning. *Springer Science+Business Media*. - Wu (1983). On the convergence properties of the EM algorithm. *Ann. Statist.*. - Goodfellow et al. (2016). Deep Learning. *MIT Press*. ## Cite this entry ```bibtex @misc{dictml_training, author = {Jung, Alexander}, editor = {Olioumtsevits, Konstantina and Schnoor, Ekkehard}, title = {training}, howpublished = {Dictionary of Applied Machine Learning (course edition)}, year = {2026}, doi = {10.5281/zenodo.21569296}, note = {ISBN 978-952-64-3013-3, CC BY 4.0, retrieved 2026-11-11}, url = {https://dictionaryofml.org/terms/training.html} } ```