interpretability.py, which recomputes what the entry states: the script block by block · open in ColabDictionary of Applied Machine Learning: 114 machine-learning terms, each with a definition, a figure, citations and a Python demo · All terms · Demos · Start here
Updated on 2026-10-29
Readiness 100/100
See also explainability explainable artificial intelligence interpretable machine learning trustworthy artificial intelligence regularization transparency
The interpretability of a machine learning (ML) method is the extent to which a human user can comprehend the computational process it carries out. Interpretability is closely related to predictability: the user should be able to anticipate the predictions of the method on a test set. Interpretability differs from explainability, which concerns understanding specific predictions with the help of provided explanations.
interpretability.py, which recomputes what the entry states: the script block by block · open in ColabP-costConsider a training set of monthly sales and a linear hypothesis learned from it. Computing next month's prediction from that hypothesis takes a slope, an intercept, and one multiplication, and a user can carry the computation out on paper. With $d$ features, the fitted linear map has one weight per feature and the prediction takes $d$ multiplications, still a computation the user can carry out. The same prediction delivered by a deep net arrives with no account a user could follow. Interpretability is what separates the two.
A machine learning (ML) method is interpretable for a human user if they can comprehend the computational process of the method. The international terminology standard for artificial intelligence (AI) does not define interpretability (International Organization for Standardization and International Electrotechnical Commission, 2022). Its closest notion is predictability, the property of an artificial intelligence system (AI system) that enables reliable assumptions by users about the predictions delivered by an ML method (International Organization for Standardization and International Electrotechnical Commission, 2022, Sect. 3.5.8 and 5.15.7).
P-weakerComprehension of a computation cannot be observed directly, so interpretability is judged through predictability: the user anticipates the predictions the method delivers, and the evaluations proposed in the literature measure how often those anticipations are right (Chen et al., 2018; Colin et al., 2022; Doshi-Velez and Kim, 2017; Hase and Bansal, 2020). Predictability is weaker than interpretability: a method whose computation the user can follow is predictable to that user, but a method whose predictions the user anticipates need not be one whose computation they could carry out. The terminology standard makes the same point: a user may rely on the predictions of an AI system without being able to comprehend how the method produced them (International Organization for Standardization and International Electrotechnical Commission, 2022, Sect. 5.15.7).
P-decompFig. 1 depicts a test of
predictability. A training set is used by two different methods
that learn hypotheses $\widehat{h}$ and
$\widehat{h}'$. The user anticipates the predictions
on a test set, providing a pseudo-label for each of its
data points, by visually extrapolating the linear trend of
the training set. The predictions of $\widehat{h}$
agree with these anticipations: the method producing
$\widehat{h}$ is predictable to the user. The
predictions of $\widehat{h}'$ deviate from them: the
method producing $\widehat{h}'$ fails the test of
predictability and therefore cannot be one whose computation the
user comprehends. The agreement alone does not establish
interpretability, since the user anticipated the predictions
of $\widehat{h}$ without carrying out its computation. The
method producing $\widehat{h}$ is nevertheless also
interpretable to a user familiar with the concept of a
linear map: such a user could comprehend the computation of
$\widehat{h}$.
P-explainThe AI Risk Management Framework of the US National Institute of Standards and Technology distinguishes interpretability from explainability (National Institute of Standards and Technology, 2023), the property pursued by explainable artificial intelligence (XAI) (Gunning and Aha, 2019). In contrast to interpretability, explainability requires that an explanation is provided along with each prediction. These explanations may take the form of saliency maps or reference examples from the training set. A challenge for XAI methods is that a computed explanation can be unfaithful to the learned hypothesis and thereby mislead the user. An interpretable machine learning (interpretable ML) method avoids this: the account of its computation is the computation itself, so the account cannot be unfaithful to it (Rudin, 2019).
See also: explainability, explainable artificial intelligence, interpretable machine learning, trustworthy artificial intelligence, regularization, local interpretable model-agnostic explanations, mechanistic interpretability, transparency.
@misc{dictml_interpretability,
author = {Jung, Alexander},
editor = {Olioumtsevits, Konstantina and Schnoor, Ekkehard},
title = {interpretability},
howpublished = {Dictionary of Applied Machine Learning (course edition)},
year = {2026},
doi = {10.5281/zenodo.21569296},
note = {ISBN 978-952-64-3013-3, CC BY 4.0, retrieved 2026-11-04},
url = {https://dictionaryofml.org/terms/interpretability.html}
}