# data imputation *Entry of the [Dictionary of Applied Machine Learning](https://dictionaryofml.org) (course edition), by Alexander Jung, Aalto University. CC BY 4.0. Updated 2026-11-08. [HTML](https://dictionaryofml.org/terms/dataimputation.html) · [typeset PDF](https://dictionaryofml.org/terms/dataimputation.pdf) · [LaTeX](https://dictionaryofml.org/terms/dataimputation.tex) · [Python demo](https://dictionaryofml.org/terms/dataimputation.py) · [notebook](https://dictionaryofml.org/terms/dataimputation.ipynb)* Math is LaTeX with the book's macros expanded, so no notation package is needed to read it. ## Abstract Data imputation replaces missing attribute values of data points by estimates computed from the attribute values that were recorded. In tabular data a missing value is an empty cell of the table, and imputation fills the cell. It is one response to missing data, alongside discarding the incomplete data points. The filling is a machine learning problem: the missing value is the label of a data point, the recorded values beside it are its features, and the complete data points form a training set on which a hypothesis map (an imputer) is learned. ## Definition The weather station Krems an der Donau records, for each day, the minimum temperature, the precipitation, and the maximum temperature. The year $2024$ is therefore a table with one row per day and one column per quantity (GeoSphere Austria, 2026). This table is tabular data. A day on which the thermometer failed leaves the maximum-temperature cell of its row empty, while the table keeps its shape (Fig. 1). Data imputation fills such a cell: it replaces missing attribute values of data points, features or labels, by estimates computed from the attribute values that were recorded (Little and Rubin, 2019). It is one response to missing data, alongside discarding the incomplete data points or using a method that tolerates the gaps (Little and Rubin, 2019). ![Figure 1 of the entry dataimputation](https://dictionaryofml.org/terms/fig/dataimputation-fig1.svg) *Figure 1: Four of the $366$ rows of the table of daily weather measurements recorded in $2024$, July 9–12. Each row is a data point, namely one day; the headers abbreviate the minimum temperature (min. temp.), the precipitation (precip.), and the maximum temperature (max. temp.), and the special value $-1.0$ of the precipitation column records that no precipitation was observed. The maximum-temperature cells of July 10 and 11 are empty. The light gray column holds the feature and the dark gray column the label of a day, so the two complete rows shown, and the other complete days, supply both, while the two empty cells are the labels to be imputed* The filling is a machine learning problem (Abayomi et al., 2008). The empty cell is the label of its row, and the recorded cells of the same row are its features. The complete rows form a training set, since each one supplies both a feature vector and the label that belongs with it. A hypothesis map, an imputer, is learned from them by empirical risk minimization. Applying that hypothesis to an incomplete row yields the estimate that fills its cell. The simplest imputation fills every empty cell of a column with the sample mean of the recorded cells of that column, which is the hypothesis that ignores the feature vector (Little and Rubin, 2019). A numerical experiment takes $15$ days spread over the year and empties the maximum-temperature cells of $3$ of them. From the $12$ complete rows, it learns a linear regression hypothesis that maps the minimum temperature of a day to its maximum temperature. The emptied cells are then filled with its predictions, shown as a scatterplot in Fig. 2. The removed values are known, so the filling can be evaluated. The average squared error loss on the $3$ cells, in squared degrees Celsius, is $17.6$ for the learned hypothesis and $150.2$ for the column average. ![Figure 2 of the entry dataimputation](https://dictionaryofml.org/terms/fig/dataimputation-fig2.svg) *Figure 2: Scatterplot of $15$ days of the table in Fig. 1, spread over the year, with the minimum temperature of a day as its feature and the maximum temperature as its label. The $12$ complete days are gray circles, and the solid line is the linear regression hypothesis learned from them. For each of the $3$ emptied cells, a cross marks the removed value and an open square the imputed value* A missing value can sit among other missing values, so that the feature vector of its row is itself incomplete. Rectangular patches lost from an aerial photograph are such a case. Each pixel is a data point, its features are the colors of the eight pixels around it, and its label is its own color, so a lost pixel inside a patch has lost neighbors as well. The filling then proceeds in sweeps: the missing values start at the sample mean of the recorded ones, and each sweep re-estimates them from the values that their neighbors currently hold, until the values no longer change. For a linear model, one sweep maps the vector ${\mathbf x}$ of missing values to ${\mathbf M} {\mathbf x} + {\mathbf b}$, with a matrix ${\mathbf M}$ built from the weights of the hypothesis. Two runs started at ${\mathbf x}$ and ${\mathbf x}'$ differ after $t$ sweeps by ${\mathbf M}^{t} ({\mathbf x} - {\mathbf x}')$, so the sweeps converge from every start if and only if ${\mathbf M}^{t} \to \mathbf{0}$, i.e., if and only if the spectral radius $\rho({\mathbf M})$ is below one (Horn and Johnson, 2013, Thm. 5.6.12), and the limit is then the unique fixed point of the sweep. A numerical experiment fills the lost pixels by such sweeps. For the hypothesis learned from the intact pixels by least squares, $\rho({\mathbf M}) = 2.72$ and the sweeps diverge. The penalty term of ridge regression shrinks the weights: with the regularization parameter $\alpha = 0.03$, the hypothesis learned by ridge regression gives $\rho({\mathbf M}) = 0.84$, and the sweeps converge. With that hypothesis, the average squared error loss on the lost pixels is half that of the baseline that fills every one of them with the average color, and the boundaries where the colors change sharply come out blurred. An imputed value is an estimate and not a measurement, and a method trained on imputed data points cannot tell the two apart. Part of the training set was produced by a hypothesis rather than observed, so a training error that averages over imputed cells scores a hypothesis partly against the predictions of the imputer rather than against measurements; for the imputer itself, every filled cell contributes zero loss. A single filled value also carries no information about the uncertainty of the estimate (Hastie et al., 2009, Sect. 9.6; Little and Rubin, 2019). Recording which values were imputed, and keeping them out of the test set, keeps the estimate of the risk free of values that a hypothesis produced. **Synonyms:** imputation, missing-value imputation. **See also:** missing data, [tabular data](https://dictionaryofml.org/terms/tabulardata.html), [data point](https://dictionaryofml.org/terms/datapoint.html), [feature](https://dictionaryofml.org/terms/feature.html), [label](https://dictionaryofml.org/terms/label.html), [feature vector](https://dictionaryofml.org/terms/featurevec.html), [training set](https://dictionaryofml.org/terms/trainset.html), [empirical risk minimization](https://dictionaryofml.org/terms/erm.html), [linear regression](https://dictionaryofml.org/terms/linreg.html), squared error loss, [baseline](https://dictionaryofml.org/terms/baseline.html), [test set](https://dictionaryofml.org/terms/testset.html). ## References - GeoSphere Austria (2026). GeoSphere Austria climate station archive. - Little and Rubin (2019). Statistical Analysis with Missing Data. *Wiley*. - Abayomi et al. (2008). Diagnostics for Multivariate Imputations. *J. Roy. Statist. Soc.: Ser. C (Appl. Statist.)*. - Horn and Johnson (2013). Matrix Analysis. *Cambridge Univ. Press*. - Hastie et al. (2009). The Elements of Statistical Learning: Data Mining, Inference, and Prediction. *Springer Science+Business Media*. ## Cite this entry ```bibtex @misc{dictml_dataimputation, author = {Jung, Alexander}, editor = {Olioumtsevits, Konstantina and Schnoor, Ekkehard}, title = {data imputation}, howpublished = {Dictionary of Applied Machine Learning (course edition)}, year = {2026}, doi = {10.5281/zenodo.21569296}, note = {ISBN 978-952-64-3013-3, CC BY 4.0, retrieved 2026-11-11}, url = {https://dictionaryofml.org/terms/dataimputation.html} } ```