Dictionary of Applied Machine Learning: 114 machine-learning terms, each with a definition, a figure, citations and a Python demo · All terms · Demos · Start here

interpretable machine learning (interpretable ML)

Updated on 2026-11-04

Readiness 100/100

▶ Run the Python demo Open in Colab Typeset PDF LaTeX source Markdown Cite this entry

See also interpretability explainability explainable artificial intelligence decision tree

Interpretable machine learning (interpretable ML) refers to machine learning (ML) methods whose hypothesis space contains only hypotheses that a human user can comprehend directly, such as sparse linear models or shallow decision trees. Such methods let a user see how a prediction changes when the features of a data point change, and how the predictions change when a different training set is used. An interpretable hypothesis is its own explanation; post hoc explanations of the predictions of an opaque learned hypothesis can instead be unfaithful. Restricting the hypothesis space to interpretable hypotheses also acts as a form of regularization. A penalty term can pick a comprehensible hypothesis out of a hypothesis space that mostly holds others: least absolute shrinkage and selection operator (Lasso) sets most weights of a linear model to zero, leaving a hypothesis with few nonzero weights, and explainable empirical risk minimization (EERM) penalizes deviations from the predictions one user supplies during training.

Figure produced by the Python demo of the entry interpretable machine learning (interpretable ML)
Figure written by the entry's Python demo interpretableml.py, which recomputes what the entry states: the script block by block · open in Colab

Definition

A physician reads a rule that predicts a patient's risk from a handful of measurements and can check every step of it by hand. Interpretable ML refers to machine learning (ML) methods whose hypothesis space contains only such hypotheses, which a human user can comprehend directly (see interpretability). Such methods let a user follow both what a learned hypothesis does to its input and how training produced it. An interpretable ML method allows the user to understand how a prediction changes when the features of a data point change, and how the predictions change when a different training set is used. Thus, interpretable ML methods require suitable choices for the hypothesis space and the training algorithm.

B-treeExamples of interpretable ML include empirical risk minimization (ERM) with linear models that combine a small number of meaningful features, shallow decision trees whose predictions follow from a few explicit tests, generalized additive models (GAMs), and lists of decision rules (Molnar, 2025). Fig. 1 depicts a decision tree for medical triage in a hospital. This system involves two tests of vital signs, allowing the staff to trace every prediction by reading the tree.

Figure 1 of the entry interpretableml
Figure 1: An interpretable hypothesis: a shallow decision tree for medical triage. Every prediction follows from at most two explicit tests of the features, so a human user can trace and anticipate the predictions
An interpretable hypothesis makes the effect of feature changes explicit. For a linear model $h^{({\mathbf w})}({\mathbf x}) = {\mathbf w}^{\top}{\mathbf x}$, changing the feature $x_{j}$ by an amount $\Delta$ changes the prediction by exactly $w_{j} \Delta$. Each weight states how strongly one feature affects the prediction. For the decision tree of Fig. 1, a prediction changes only when a feature crosses one of the explicit test thresholds. A patient at $38.5\,^{\circ}\mathrm{C}$ with a heart rate of $110/\mathrm{min}$ is routine; a rise to $38.9\,^{\circ}\mathrm{C}$ leaves that prediction unchanged, while a rise to $39.5\,^{\circ}\mathrm{C}$ turns it urgent.

B-effectAn interpretable ML method also makes the effect of training set changes traceable. This dependence is studied under stability, which formalizes an ML method as a map from a training set to a learned hypothesis. For linear regression, this map is available in closed form through the normal equations, so the user can compute how a perturbed label in the training set shifts the predictions (see linear regression).

An interpretable hypothesis is its own explanation: no separate explanation is constructed for it. The explanation delivered with a prediction can then be a description of the learned hypothesis itself: the weights of a linear model that combines few features, or the flow chart of a shallow decision tree as in Fig. 1. A hypothesis space containing only such hypotheses is called intrinsically interpretable (Molnar, 2025). This contrasts with explainable artificial intelligence (XAI), which constructs post hoc explanations for the predictions of a possibly opaque learned hypothesis. For high-stakes applications, such as the medical triage of Fig. 1, using an interpretable model can be preferable to explaining an opaque model, since post hoc explanations can be unfaithful to the hypothesis they explain (Rudin, 2019).

B-limeRestricting $\mathcal{H}$ to interpretable hypotheses is model pruning, one of the three routes that regularization distinguishes. The penalty-term route gives an interpretable learned hypothesis without restricting the hypothesis space beforehand. A penalty term added to the objective function of ERM penalizes the hypotheses a user cannot follow, so training picks a comprehensible one out of a hypothesis space that mostly holds the others. Least absolute shrinkage and selection operator (Lasso) does this for linear models, which are no more comprehensible than a deep net once they combine hundreds of features. Lasso uses the $\ell_{1}$-norm $\left\lVert {\mathbf w} \right\rVert_{1}$ of the weights ${\mathbf w}$ as the penalty term, which sets most weights to exactly zero (Hastie et al., 2009, Sect. 3.4.2). The learned hypothesis is then a weighted sum of a few features, so a user can name the features the prediction uses and how much each one changes it.

B-eermA method close to this route, but not interpretable ML, is explainable empirical risk minimization (EERM), which extends the idea to the training of an arbitrary model. Its penalty term is built from the predictions that one specific user supplies for the data points of the training set (see EERM). It therefore delivers explainability to that user rather than interpretability, and it leaves the hypothesis space as it is. The user predictions need not be elicited one data point at a time. They might be obtained as the predictions of a simpler proxy model that the user considers interpretable, so that the penalty term measures the deviation from that proxy. In Fig. 2, the user predictions are obtained from a simple linear model fitted to the training set.

Figure 2 of the entry interpretableml
Figure 2: The idea of EERM (Zhang et al., 2024): training a high-dimensional model (polynomials of degree $10$) with a penalty term that measures the deviation of the predictions of the learned hypothesis from the user predictions. These user predictions are obtained from a simple surrogate linear model (dotted line). ERM without a penalty term (thin solid line) overfits the training set, while EERM delivers a hypothesis (dashed line) that stays close to the user predictions. Data generated by pythondemos/interpretableml.py

See also: interpretability, explainability, explainable artificial intelligence, decision tree, decision rule, generalized additive model, least absolute shrinkage and selection operator, stability, explainable empirical risk minimization.

References

  1. Molnar (2025). Interpretable Machine Learning: A Guide for Making Black Box Models Explainable. Ebook. christophm.github.io/interpretable-ml-book/
  2. Rudin (2019). Stop explaining black box machine learning models for high-stakes decisions and use interpretable models instead. Nature Mach. Intell.. doi.org/10.1038/s42256-019-0048-x
  3. Hastie et al. (2009). The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer Science+Business Media. doi.org/10.1007/978-0-387-84858-7
  4. Zhang et al. (2024). Explainable empirical risk minimization. Neural Comput. Appl.. doi.org/10.1007/s00521-023-09269-3

Cite this entry

@misc{dictml_interpretableml,
  author = {Jung, Alexander},
  editor = {Olioumtsevits, Konstantina and Schnoor, Ekkehard},
  title = {interpretable machine learning (interpretable ML)},
  howpublished = {Dictionary of Applied Machine Learning (course edition)},
  year = {2026},
  doi = {10.5281/zenodo.21569296},
  note = {ISBN 978-952-64-3013-3, CC BY 4.0, retrieved 2026-11-04},
  url = {https://dictionaryofml.org/terms/interpretableml.html}
}