Dictionary of Applied Machine Learning

online learning

Updated on 2026-08-28

Typeset PDF version — the authoritative form of this entry

Online learning refers to machine learning (ML) methods that process data sequentially: at each discrete time instant a new data point arrives and the method updates its current model parameters, in contrast with solving empirical risk minimization (ERM) once over a fixed training set. The canonical update is a gradient step on the loss of the newest data point, which is online gradient descent (online GD). For data points that are realizations of independent and identically distributed (i.i.d.) random variables (RVs), the expected update is a gradient step on the risk, and online GD coincides with stochastic gradient descent (SGD) with batch size one. Performance is measured by the regret, the accumulated loss minus that of the best fixed model parameters in hindsight. With a suitably decaying learning rate, the average regret per round of online GD vanishes as the number of rounds grows.

Definition

A Finnish Meteorological Institute (FMI) weather station delivers one new data point every evening: the day's morning minimum and its maximum daytime temperature. A method that first collects a complete training set never starts forecasting. Online learning instead updates the hypothesis every day, using the newest data point (Jung, 2022, Sect. 4.7).

Online learning refers to machine learning (ML) methods that process data sequentially: data points $\datapoint^{(\timeidx)}$ arrive at discrete time instants $\timeidx = 1, 2, \ldots$, and at each instant the method updates its current model parameters $\weights^{(\timeidx)}$, or hypothesis $\hypothesis^{(\timeidx)}$, using $\datapoint^{(\timeidx)}$ (Fig. 1). This contrasts with solving empirical risk minimization (ERM) once over a fixed training set. The time index is also the iteration index: one new data point means one update.

Figure 1 of the entry onlinelearning
Figure 1: One round of online learning. The model parameters $\weights^{(\timeidx)}$ are updated as soon as the data point $\datapoint^{(\timeidx)}$ of time instant $\timeidx$ arrives, and the updated model parameters $\weights^{(\timeidx+1)}$ enter the next round. In online GD the update is a gradient step on $\lossfunc{\datapoint^{(\timeidx)}}{\cdot}$
The canonical update is a gradient step on the loss of the newest data point, \[ \weights^{(\timeidx+1)} = \weights^{(\timeidx)} - \lrate^{(\timeidx)} \nabla_{\weights} \lossfunc{\datapoint^{(\timeidx)}}{\weights^{(\timeidx)}} \text{,} \] which is online gradient descent (online GD). The iteration applies no single fixed operator: the update at time $\timeidx$ is the gradient step operator of the newest loss component, which changes from round to round, so a fixed point interpretation with one operator is not meaningful here. For data points that are realizations of independent and identically distributed (i.i.d.) random variables (RVs), the expected update is a gradient step on the risk, whose fixed points are the stationary points of the risk; online GD then coincides with stochastic gradient descent (SGD) with batch size one.

The quality of an online method is measured in hindsight. After $\nriter$ rounds, the accumulated loss $\sum_{\timeidx=1}^{\nriter} \lossfunc{\datapoint^{(\timeidx)}}{\weights^{(\timeidx)}}$ is compared with the accumulated loss of the best fixed model parameters chosen after all data points have been seen; the difference is the regret. With a suitably decaying learning rate, online GD keeps the average regret per round vanishing as $\nriter$ grows (Hazan, 2022).

For linear regression on the temperature stream, the model parameters that minimize the average squared error loss over all days up to time $\timeidx$ need not be recomputed from scratch each day: the solution at time $\timeidx + 1$ reuses the matrix computations of time $\timeidx$ (Jung, 2022, Sect. 4.7).

See also: online gradient descent, online algorithm, stochastic gradient descent, gradient descent, gradient step, empirical risk minimization, regret, risk, linear regression.

References

  1. Jung (2022). Machine Learning: The Basics. Springer Nature. doi.org/10.1007/978-981-16-8193-6
  2. Hazan (2022). Introduction to Online Convex Optimization. MIT Press. sites.google.com/view/intro-oco/

Cite this entry

@misc{dictml_onlinelearning,
  author = {Jung, Alexander and Olioumtsevits, Konstantina and Schnoor, Ekkehard},
  title = {online learning},
  howpublished = {Dictionary of Applied Machine Learning (course edition)},
  year = {2026},
  doi = {10.5281/zenodo.21569296},
  note = {ISBN 978-952-64-3013-3, CC BY 4.0, retrieved 2026-09-19},
  url = {https://dictionaryofml.org/terms/onlinelearning.html}
}