# interpretable machine learning (interpretable ML) *Entry of the [Dictionary of Applied Machine Learning](https://dictionaryofml.org) (course edition), by Alexander Jung, Aalto University. CC BY 4.0. Updated 2026-11-04. [HTML](https://dictionaryofml.org/terms/interpretableml.html) · [typeset PDF](https://dictionaryofml.org/terms/interpretableml.pdf) · [LaTeX](https://dictionaryofml.org/terms/interpretableml.tex) · [Python demo](https://dictionaryofml.org/terms/interpretableml.py) · [notebook](https://dictionaryofml.org/terms/interpretableml.ipynb)* Math is LaTeX with the book's macros expanded, so no notation package is needed to read it. ## Abstract Interpretable machine learning (interpretable ML) refers to machine learning methods whose hypothesis space contains only hypotheses that a human user can comprehend directly, such as sparse linear models or shallow decision trees. Such methods let a user see how a prediction changes when the features of a data point change, and how the predictions change when a different training set is used. An interpretable hypothesis is its own explanation; post hoc explanations of the predictions of an opaque learned hypothesis can instead be unfaithful. Restricting the hypothesis space to interpretable hypotheses also acts as a form of regularization. A penalty term can pick a comprehensible hypothesis out of a hypothesis space that mostly holds others: least absolute shrinkage and selection operator sets most weights of a linear model to zero, leaving a hypothesis with few nonzero weights, and explainable empirical risk minimization penalizes deviations from the predictions one user supplies during training. ## Definition A physician reads a rule that predicts a patient's risk from a handful of measurements and can check every step of it by hand. Interpretable ML refers to machine learning methods whose hypothesis space contains only such hypotheses, which a human user can comprehend directly (see interpretability). Such methods let a user follow both what a learned hypothesis does to its input and how training produced it. An interpretable machine learning method allows the user to understand how a prediction changes when the features of a data point change, and how the predictions change when a different training set is used. Thus, interpretable machine learning methods require suitable choices for the hypothesis space and the training algorithm. Examples of interpretable machine learning include empirical risk minimization with linear models that combine a small number of meaningful features, shallow decision trees whose predictions follow from a few explicit tests, GAMs, and lists of decision rules (Molnar, 2025). Fig. 1 depicts a decision tree for medical triage in a hospital. This system involves two tests of vital signs, allowing the staff to trace every prediction by reading the tree. ![Figure 1 of the entry interpretableml](https://dictionaryofml.org/terms/fig/interpretableml-fig1.svg) *Figure 1: An interpretable hypothesis: a shallow decision tree for medical triage. Every prediction follows from at most two explicit tests of the features, so a human user can trace and anticipate the predictions* An interpretable hypothesis makes the effect of feature changes explicit. For a linear model $h^{({\mathbf w})}({\mathbf x}) = {\mathbf w}^{\top}{\mathbf x}$, changing the feature $x_{j}$ by an amount $\Delta$ changes the prediction by exactly $w_{j} \Delta$. Each weight states how strongly one feature affects the prediction. For the decision tree of Fig. 1, a prediction changes only when a feature crosses one of the explicit test thresholds. A patient at $38.5\,^{\circ}\mathrm{C}$ with a heart rate of $110/\mathrm{min}$ is routine; a rise to $38.9\,^{\circ}\mathrm{C}$ leaves that prediction unchanged, while a rise to $39.5\,^{\circ}\mathrm{C}$ turns it urgent. An interpretable machine learning method also makes the effect of training set changes traceable. This dependence is studied under stability, which formalizes a machine learning method as a map from a training set to a learned hypothesis. For linear regression, this map is available in closed form through the normal equations, so the user can compute how a perturbed label in the training set shifts the predictions (see linear regression). An interpretable hypothesis is its own explanation: no separate explanation is constructed for it. The explanation delivered with a prediction can then be a description of the learned hypothesis itself: the weights of a linear model that combines few features, or the flow chart of a shallow decision tree as in Fig. 1. A hypothesis space containing only such hypotheses is called intrinsically interpretable (Molnar, 2025). This contrasts with explainable artificial intelligence, which constructs post hoc explanations for the predictions of a possibly opaque learned hypothesis. For high-stakes applications, such as the medical triage of Fig. 1, using an interpretable model can be preferable to explaining an opaque model, since post hoc explanations can be unfaithful to the hypothesis they explain (Rudin, 2019). Restricting $\mathcal{H}$ to interpretable hypotheses is model pruning, one of the three routes that regularization distinguishes. The penalty-term route gives an interpretable learned hypothesis without restricting the hypothesis space beforehand. A penalty term added to the objective function of empirical risk minimization penalizes the hypotheses a user cannot follow, so training picks a comprehensible one out of a hypothesis space that mostly holds the others. Least absolute shrinkage and selection operator does this for linear models, which are no more comprehensible than a deep net once they combine hundreds of features. Least absolute shrinkage and selection operator uses the $\ell_{1}$-norm $\left\lVert {\mathbf w} \right\rVert_{1}$ of the weights ${\mathbf w}$ as the penalty term, which sets most weights to exactly zero (Hastie et al., 2009, Sect. 3.4.2). The learned hypothesis is then a weighted sum of a few features, so a user can name the features the prediction uses and how much each one changes it. A method close to this route, but not interpretable machine learning, is explainable empirical risk minimization, which extends the idea to the training of an arbitrary model. Its penalty term is built from the predictions that one specific user supplies for the data points of the training set (see explainable empirical risk minimization). It therefore delivers explainability to that user rather than interpretability, and it leaves the hypothesis space as it is. The user predictions need not be elicited one data point at a time. They might be obtained as the predictions of a simpler proxy model that the user considers interpretable, so that the penalty term measures the deviation from that proxy. In Fig. 2, the user predictions are obtained from a simple linear model fitted to the training set. ![Figure 2 of the entry interpretableml](https://dictionaryofml.org/terms/fig/interpretableml-fig2.svg) *Figure 2: The idea of explainable empirical risk minimization (Zhang et al., 2024): training a high-dimensional model (polynomials of degree $10$) with a penalty term that measures the deviation of the predictions of the learned hypothesis from the user predictions. These user predictions are obtained from a simple surrogate linear model (dotted line). Empirical risk minimization without a penalty term (thin solid line) overfits the training set, while explainable empirical risk minimization delivers a hypothesis (dashed line) that stays close to the user predictions. Data generated by pythondemos/interpretableml.py* **See also:** [interpretability](https://dictionaryofml.org/terms/interpretability.html), [explainability](https://dictionaryofml.org/terms/explainability.html), [explainable artificial intelligence](https://dictionaryofml.org/terms/xaiterm.html), [decision tree](https://dictionaryofml.org/terms/decisiontree.html), decision rule, generalized additive model, least absolute shrinkage and selection operator, stability, explainable empirical risk minimization. ## References - Zhang et al. (2024). Explainable empirical risk minimization. *Neural Comput. Appl.*. - Molnar (2025). Interpretable Machine Learning: A Guide for Making Black Box Models Explainable. *Ebook*. - Rudin (2019). Stop explaining black box machine learning models for high-stakes decisions and use interpretable models instead. *Nature Mach. Intell.*. - Hastie et al. (2009). The Elements of Statistical Learning: Data Mining, Inference, and Prediction. *Springer Science+Business Media*. ## Cite this entry ```bibtex @misc{dictml_interpretableml, author = {Jung, Alexander}, editor = {Olioumtsevits, Konstantina and Schnoor, Ekkehard}, title = {interpretable machine learning (interpretable ML)}, howpublished = {Dictionary of Applied Machine Learning (course edition)}, year = {2026}, doi = {10.5281/zenodo.21569296}, note = {ISBN 978-952-64-3013-3, CC BY 4.0, retrieved 2026-11-04}, url = {https://dictionaryofml.org/terms/interpretableml.html} } ```