trustAI.py, which recomputes what the entry states: the script block by block · open in ColabDictionary of Applied Machine Learning: 109 machine-learning terms, each with a definition, a figure, citations and a Python demo · All terms · Demos · Start here
Updated on 2026-10-28
Readiness 70/100
See also transparency explainability fairness empirical risk minimization
Trustworthy artificial intelligence (trustworthy AI) names an artificial intelligence system (AI system) that meets requirements beyond accuracy: human oversight, robustness, privacy, transparency, fairness, societal well-being, and accountability. The seven key requirements of the EU Ethics Guidelines are stated for the AI system as a whole, including the people and the organization around the machine learning (ML) method inside it. Five of them constrain the components of empirical risk minimization (ERM): the training set, the model, the loss function, the algorithm, and the learned hypothesis. Each such requirement is a measurable property of that hypothesis or of the method that learned it: a bound on the change of a prediction under a perturbation of the feature vector, the change of the prediction when only a sensitive attribute is flipped, the counterfactual that explains a rejection, or the share of borderline decisions passed to a person. The EU AI Act makes several of these requirements binding for a high-risk artificial intelligence system (high-risk AI system), and they can conflict, so trustworthiness is a set of measured properties balanced for the application rather than a single number.
trustAI.py, which recomputes what the entry states: the script block by block · open in ColabConsider a bank that uses an artificial intelligence system (AI system) to decide on loan applications. The system applies a hypothesis $\widehat{h}$, learned from the records of completed loans, to the feature vector ${\mathbf x}$ of an applicant, which collects the income, the gender, and further attributes. The loan is approved when the predicted creditworthiness $\widehat{h}({\mathbf x})$ exceeds a threshold $\tau$. The bank, its supervisor, and the applicant rely on these decisions only if the system meets requirements beyond accuracy. A small change of the income must not flip the decision, the gender alone must not flip it either, a rejected applicant must learn why, and a loan officer must be able to overrule the system. Trustworthy AI names an AI system that meets such requirements.
The Ethics Guidelines of the European Commission's High-Level Expert Group state seven key requirements (KRs) (High-Level Expert Group on Artificial Intelligence, 2019): human agency and oversight (KR1), technical robustness and safety (KR2), privacy and data governance (KR3), transparency (KR4), diversity, nondiscrimination, and fairness (KR5), societal and environmental well-being (KR6), and accountability (KR7). The accompanying assessment list turns each requirement into questions that the provider and the deployer of the AI system answer about their system (High-Level Expert Group on Artificial Intelligence, 2020).
The requirements are stated for an AI system as a whole, which includes the data pipeline, the user interface, and the organization that operates it, and not only for the machine learning (ML) method inside it. KR6 and KR7 concern the purpose and the governance of the system and have no counterpart in the ML method. KR1 to KR5, in contrast, constrain the components of empirical risk minimization (ERM): the training set $\mathcal{D}^{(\rm train)}$, the model $\mathcal{H}$, the loss function, and an algorithm that minimizes the average loss over $\mathcal{D}^{(\rm train)}$, as well as the learned hypothesis $\widehat{h}$ that results (Pfau and Jung, 2024). Each of these requirements can be stated as a measurable property of $\widehat{h}$ or of the method that learned it, and the design choices for the components of ERM are the means to meet it.
Fig. 1 places KR1, KR2, KR4, and KR5 in the credit-scoring example. The horizontal axis is a one-dimensional sketch of the feature vector ${\mathbf x}$: the income, the gender, and the further attributes of an applicant collapsed into one coordinate. The vertical axis is the creditworthiness, the label $y$. The filled circles are the training set $\mathcal{D}^{(\rm train)}$ of $m = 5$ completed loans, the solid curve the hypothesis $\widehat{h}$ learned by minimizing the average loss over $\mathcal{D}^{(\rm train)}$, and the dotted line the approval threshold $\tau = 3.8$. The applicant with feature vector ${\mathbf x}'$ (open square) obtains the predicted creditworthiness $\widehat{h}({\mathbf x}') = 2.94$ and is rejected.
B-robustKR2 asks that a small perturbation of the feature vector changes the prediction little (Pfau and Jung, 2024, Sect. V). The perturbation of ${\mathbf x}'$ by $\delta = 0.5$ (horizontal arrow) moves the predicted creditworthiness from $2.94$ to $3.19$ (vertical arrow). The slope of $\widehat{h}$ never exceeds $1.225$, so no perturbation by $\delta$ changes the prediction by more than $1.225\,\delta$: the learned hypothesis is Lipschitz continuous with constant $1.225$ (see robustness). KR2 also covers perturbations of $\mathcal{D}^{(\rm train)}$, e.g., by data poisoning, against which the stability of the ML method protects.
B-fairKR5 asks that the decision does not depend on a sensitive attribute $s$ such as gender. The open triangle depicts the same applicant with only the gender flipped, the feature vector ${\mathbf x}^{\star}$. The predicted creditworthiness rises to $\widehat{h}({\mathbf x}^{\star}) = 3.93$, above $\tau$, and the loan is approved: the decision depends on the gender, which violates the counterfactual criterion of fairness. The gap $\widehat{h}({\mathbf x}^{\star}) - \widehat{h}({\mathbf x}') = 0.99$ measures the violation. It is closed by a design choice for a component of ERM: the model (excluding the sensitive attribute and the features correlated with it), the loss function (a penalty term for the gap), or $\mathcal{D}^{(\rm train)}$ (resampling the groups) (Pfau and Jung, 2024, Sect. VIII).
B-explainKR4 asks for transparency: the rejected applicant is told that an AI system was used and may obtain an explanation. The counterfactual ${\mathbf x}''$ (open diamond), at which $\widehat{h}$ reaches $\tau$, is such an explanation. It is the smallest change of the income, at fixed gender, that flips the decision, and it tells the applicant the income at which the loan would be approved. The gender flip ${\mathbf x}^{\star}$ also flips the decision, but it is not a change the applicant can make, which is what makes it a violation of KR5 rather than an explanation.
B-oversightKR1 asks that a person can oversee and overrule the decisions.
The shaded band marks predicted creditworthiness within
$\varepsilon = 0.3$ of $\tau$. Applications in this band are
passed to a loan officer, who decides; one of the five loans in
$\mathcal{D}^{(\rm train)}$ falls into it. A wider band passes more applications
to the officer and leaves fewer borderline decisions to the system
(see human oversight). KR3 asks that the records of completed
loans, which are personal data, are protected during
training and cannot be recovered from $\widehat{h}$;
differential privacy (DP) and federated learning (FL) are design choices for the
algorithm and for $\mathcal{D}^{(\rm train)}$ to that end (see
data governance).
See also: artificial intelligence system, EU AI Act, high-risk artificial intelligence system, provider, deployer, human oversight, robustness, stability, data poisoning, data governance, differential privacy, transparency, explainability, counterfactual, fairness, sensitive attribute, empirical risk minimization.
@misc{dictml_trustAI,
author = {Jung, Alexander},
editor = {Olioumtsevits, Konstantina and Schnoor, Ekkehard},
title = {trustworthy artificial intelligence (trustworthy AI)},
howpublished = {Dictionary of Applied Machine Learning (course edition)},
year = {2026},
doi = {10.5281/zenodo.21569296},
note = {ISBN 978-952-64-3013-3, CC BY 4.0, retrieved 2026-10-28},
url = {https://dictionaryofml.org/terms/trustAI.html}
}