Dictionary of Applied Machine Learning

bootstrap aggregating

Updated on 2026-09-06

Typeset PDF version — the authoritative form of this entry

Bagging is an ensemble technique that trains each base learner on a resampled copy of the training set, typically drawn by bootstrap sampling. The learned hypotheses are aggregated by a majority vote in classification or by averaging in regression. Averaging over many base learners reduces the variance of the final prediction: fluctuations caused by the particular training set tend to cancel. A random forest applies bagging to decision trees.

Definition

A single decision tree, fit to a bank's customer records to predict loan defaults, changes its predictions noticeably when a handful of records are replaced. Bagging suppresses this sensitivity: it is an ensemble technique in which each base learner is trained on a resampled copy of the training set, and their predictions are aggregated (Breiman, 1996). Given the original training set $\dataset$, bagging generates perturbed copies $\widetilde{\dataset}^{(1)},\,\ldots,\, \widetilde{\dataset}^{(\numlearners)}$ (typically via bootstrap sampling) and trains one base learner on each, producing hypotheses $\learnthypothesis^{(1)},\,\ldots,\, \learnthypothesis^{(\numlearners)}$. The final hypothesis aggregates these by majority vote for classification (i.e., the class predicted by the most base learners) or averaging for regression (see Fig. 1).

Figure 1 of the entry bagging
Figure 1: Example of bagging in which three base learners use perturbations $\widetilde{\dataset}^{(1)},\,\ldots,\,\widetilde{\dataset}^{(3)}$ of the original training set $\dataset$ to learn the hypotheses $\learntlocalhypothesis{1},\,\ldots,\,\learntlocalhypothesis{3}$. The final hypothesis is obtained by aggregating these individual hypotheses via some aggregation rule $\phi_{\rm agg}$
A random forest, for example, applies bagging to decision trees: each tree is trained on a different bootstrap sample, reducing the variance of the overall prediction.

See also: ensemble, bootstrap, random forest, base learner, variance.

References

  1. Breiman (1996). Bagging predictors. Mach. Learn.. doi.org/10.1007/BF00058655

Cite this entry

@misc{dictml_bagging,
  author = {Jung, Alexander and Olioumtsevits, Konstantina and Schnoor, Ekkehard},
  title = {bootstrap aggregating},
  howpublished = {Dictionary of Applied Machine Learning (course edition)},
  year = {2026},
  doi = {10.5281/zenodo.21569296},
  note = {ISBN 978-952-64-3013-3, CC BY 4.0, retrieved 2026-09-11},
  url = {https://dictionaryofml.org/terms/bagging.html}
}