Dictionary of Applied Machine Learning
Updated on 2026-09-07
Typeset PDF version — the authoritative form of this entry
Python demo — a script that recomputes what this entry states and prints one line per check
A machine learning (ML) method is explainable if there is an effective way to explain its predictions. The method delivers an explanation along with every prediction, and that explanation is effective if it lets a human user comprehend how the features of a data point drive the prediction made for it. Explainability is always relative to a specific user or group of users: an explanation can be effective for one and useless for another. Explainability can be measured by comparing the predictions of a learned hypothesis to the anticipations of a user before and after they are provided with an explanation.
B-linA machine learning (ML) method is explainable
for a human user if the explanations it provides let the
user anticipate the predictions it delivers
(Colin et al., 2022; Jung and Nardelli, 2020). Explainability
thus includes the notion of an explanation: each
prediction is delivered with an explanation for
this specific prediction
(see Fig. 1), such as the
feature values that drove it or, for image data,
the relevant pixels.
The definition asks for the existence of an explanation, not
for a particular one: explainability is certified by
exhibiting one explanation that works, and a single
explanation the user cannot follow refutes nothing.
Explainability is relative to a specific user or group of users:
the same learned hypothesis
can be explainable for one user and inscrutable for another.
A probabilistic model for data generation provides a second measure of this predictability: the conditional differential entropy of the predictions given the anticipations (Chen et al., 2018; Jung and Nardelli, 2020). The conditional differential entropy quantifies the uncertainty that remains about the predictions once the anticipations are known, so a smaller value means that the anticipations determine the predictions more tightly. In practice, this conditional differential entropy is unknown and must be replaced by an estimator, e.g., an estimate computed from the empirical frequencies of discretized predictions and anticipations on a test set.
B-explAn explanation raises explainability when the user can
reason with it.
Fig. 2 illustrates this for a
user who reasons in terms of linear maps and anticipates the
predictions of an opaque hypothesis (a
kernel method). Without explanations, the anticipations
deviate strongly from the predictions. Given a local linear
approximation of the hypothesis around each data point
(cf. local interpretable model-agnostic explanations (LIME)), the anticipations match the predictions
almost exactly. The example presupposes that the user knows how to
apply a linear approximation: the anticipations improve only
because the user can evaluate the explanation for a
data point. A user who cannot is left where they started.
pythondemos/explainability.py
Regulation treats explainability as an ingredient of transparency. The EU AI Act requires that high-risk artificial intelligence systems (high-risk AI systems) are sufficiently transparent to enable deployers to interpret their outputs (European Parliament and Council of the European Union, 2024, Art. 13). For individual automated decisions, the right to explanation entitles an affected person to a clear and meaningful account of the role that an AI system played in the decision (European Parliament and Council of the European Union, 2024, Art. 86). The Colorado Automated Decision-Making Technology Act of 2026 requires that the deployer give the affected consumer a plain-language description of the decision and of the role the technology played in it, together with the means to request further information (Colorado General Assembly, 2026, Sect. 6-1-1704(3)).
What counts as an automated decision is drawn widely: the Court of Justice of the European Union held that a credit information agency computing a person's ability to meet future payments already makes one, when a third party draws strongly on that value (Court of Justice of the European Union, 2023).
See also: explanation, interpretability, explainable artificial intelligence, explainable empirical risk minimization, local interpretable model-agnostic explanations, transparency, right to explanation, trustworthy artificial intelligence, regularization.
@misc{dictml_explainability,
author = {Jung, Alexander},
editor = {Olioumtsevits, Konstantina and Schnoor, Ekkehard},
title = {explainability},
howpublished = {Dictionary of Applied Machine Learning (course edition)},
year = {2026},
doi = {10.5281/zenodo.21569296},
note = {ISBN 978-952-64-3013-3, CC BY 4.0, retrieved 2026-09-22},
url = {https://dictionaryofml.org/terms/explainability.html}
}