Dictionary of Applied Machine Learning
Typeset PDF version — the authoritative form of this entry
Interpretable machine learning (interpretable ML) refers to machine learning (ML) methods whose hypothesis space contains only hypotheses that a human user can comprehend directly, such as sparse linear models or shallow decision trees. Such methods maximally enable the user to understand how a prediction changes when the features of a data point change, and how the predictions change when a different training set is used. An interpretable hypothesis is its own explanation; post hoc explanations of the predictions of an opaque learned hypothesis can instead be unfaithful. Restricting the hypothesis space to interpretable hypotheses also acts as a form of regularization. Interpretable surrogates serve opaque hypotheses as well: local interpretable model-agnostic explanations (LIME) approximates them locally by sparse linear hypotheses, and explainable empirical risk minimization (EERM) penalizes the discrepancy to a simple surrogate during training.
Interpretable machine learning (ML) refers to ML methods whose hypothesis space contains only hypotheses that a human user can comprehend directly (see interpretability). Such methods maximally enable the user to understand both the input-output behavior of a learned hypothesis and the inner workings of training. Ultimately, an interpretable ML method allows the user to understand how a prediction changes when the features of a data point change, and how the predictions change when a different training set is used.
Examples include linear models that
combine a small number of meaningful features, shallow
decision trees whose predictions follow from a few
explicit tests, generalized additive models (GAMs), and lists of
decision rules (Molnar, 2025). For example, a medical triage system can use a
decision tree with two tests of vital signs: the staff can trace
every prediction by reading the tree
(see Fig. 1).
An interpretable hypothesis is its own explanation: no separate explanation machinery is required. Such a model is called intrinsically interpretable (Molnar, 2025). This contrasts with explainable artificial intelligence (XAI), which constructs post hoc explanations for the predictions of a possibly opaque learned hypothesis. For high-stakes applications, using an interpretable model is often preferable to explaining an opaque model, since post hoc explanations can be unfaithful to the hypothesis they explain (Rudin, 2019).
An explanation is faithful when it reflects the actual computation of the hypothesis it explains. For a surrogate hypothesis serving as explanation, faithfulness can be quantified by the agreement between its predictions and those of the explained hypothesis on a test set. A post hoc explanation that is faithful everywhere would coincide with the explained hypothesis itself. Any simpler explanation must therefore deviate somewhere (Rudin, 2019). Moreover, on structured (tabular) data, hypotheses learned within an interpretable model often achieve accuracy comparable to hypotheses learned within an opaque model (Rudin, 2019).
In an influential taxonomy, interpretable models are called transparent, with three degrees of transparency: simulatability (a human user can reproduce the computation of the hypothesis), decomposability (every part of the hypothesis, such as a single test in a decision tree, admits an intuitive description), and algorithmic transparency (the behavior of the training algorithm is understood) (Lipton, 2018). The two questions above map onto this taxonomy: simulatability and decomposability concern the input-output behavior of a hypothesis, while algorithmic transparency concerns the dependence of the learned hypothesis on the training set. Definitions of the underlying property, and its treatment by terminology standards, are discussed under interpretability.
Regulation uses the neighboring notion of transparency rather than defining interpretability: the EU AI Act requires that a high-risk artificial intelligence system (high-risk AI system) be designed so that its operation is sufficiently transparent to enable deployers to interpret the output of the artificial intelligence system (AI system) and use it appropriately (Parliament and Union, 2024, Art. 13). An interpretable ML method supports this requirement, since the deployer can trace every prediction back all the way to the choice of training set (see right to explanation).
Restricting the hypothesis space to interpretable hypotheses
is a form of regularization: it prunes the model and
thereby reduces the risk of overfitting. Least absolute shrinkage and selection operator (Lasso)
illustrates this connection: its penalty term favors sparse
linear models, which use few features and are therefore
easier to interpret (Hastie et al., 2009, Sect. 3.4).
Explainable empirical risk minimization (EERM) also uses a penalty term, one that targets
interpretability directly (Zhang et al., 2024). Its penalty term
measures the discrepancy between a hypothesis and a simple
surrogate, steering the training of a high-dimensional model
toward hypotheses that stay close to the surrogate
(see Fig. 2).
pythondemos/interpretableml.py
pythondemos/interpretableml.py
@misc{dictml_interpretableml,
author = {Jung, Alexander},
title = {interpretable machine learning},
howpublished = {Dictionary of Applied Machine Learning (course edition)},
year = {2026},
doi = {10.5281/zenodo.21569296},
note = {ISBN 978-952-64-3013-3, CC BY 4.0, retrieved 2026-08-06},
url = {https://dictionaryofml.org/terms/interpretableml.html}
}