# Dictionary of Applied Machine Learning

> Open (CC BY 4.0) course edition of the Dictionary of Applied Machine Learning: 68 terms, each defined precisely and cross-referenced, with citations, a typeset PDF, and (for most terms) a Python demo that recomputes what the entry states. Maintained by Alexander Jung, Aalto University. DOI 10.5281/zenodo.21569296, ISBN 978-952-64-3013-3.

Full machine-readable content: [terms.json](https://dictionaryofml.org/terms.json) (one JSON file, every entry). Each term page carries a "Cite this entry" BibTeX block. Companion code (notebooks, LaTeX notation package, MCP server): https://github.com/alexjungaalto/dictionaryofml

## Terms

- [accuracy](https://dictionaryofml.org/terms/accuracy.html): Accuracy is the fraction of correct predictions made by a hypothesis on a dataset with a finite label space, equal to one minus the average 0/1 loss.
- [artificial neural network](https://dictionaryofml.org/terms/ann.html): An artificial neural network (ANN) is a graphical (signal-flow) representation of a hypothesis that maps the features of a data point at its input to a prediction for the label at its output.
- [attention](https://dictionaryofml.org/terms/attention.html): Attention is a mechanism that models the dependencies between the tokens that make up a data point, such as the words in a sentence or the pixel patches of an image.
- [bootstrap aggregating](https://dictionaryofml.org/terms/bagging.html): Bagging is an ensemble technique that trains each base learner on a resampled copy of the training set, typically drawn by bootstrap sampling.
- [baseline](https://dictionaryofml.org/terms/baseline.html): A baseline is a reference level against which the performance of a trained model is compared.
- [boosting](https://dictionaryofml.org/terms/boosting.html): Boosting is an iterative optimization method that learns an accurate hypothesis by sequentially combining less accurate base learners, referred to as weak learners.
- [concept activation vector](https://dictionaryofml.org/terms/cav.html): A concept activation vector (CAV) represents a concept named by a user as a direction in the activations of one hidden layer of a deep net.
- [classification](https://dictionaryofml.org/terms/classification.html): Classification is the task of predicting a discrete-valued label for a given data point, based solely on its features.
- [clustering](https://dictionaryofml.org/terms/clustering.html): Clustering decomposes a dataset without labels into a small number of subsets, called clusters, such that data points within the same cluster are more similar to each other than to data points in other clusters.
- [confusion matrix](https://dictionaryofml.org/terms/cm.html): The confusion matrix of a hypothesis on a finite dataset with k label values is the k × k matrix whose entry (c, c') counts the data points with true label c and prediction c'.
- [convex](https://dictionaryofml.org/terms/convex.html): A subset of the Euclidean space is convex if it contains the line segment between any two of its points.
- [covariance matrix](https://dictionaryofml.org/terms/covmtx.html): The covariance matrix of a random vector consists of the covariances between its entries.
- [data](https://dictionaryofml.org/terms/data.html): This entry distinguishes three main uses of the term \emph{data}.
- [data leakage](https://dictionaryofml.org/terms/dataleakage.html): Data leakage is the use, during the training or the evaluation of a machine learning method, of information that is unavailable at the time a prediction must be delivered.
- [data point](https://dictionaryofml.org/terms/datapoint.html): A data point is the elementary information-carrying unit on which machine learning methods operate.
- [dataset](https://dictionaryofml.org/terms/dataset.html): A dataset is a finite collection of data points on which model training and validation are performed.
- [decision tree](https://dictionaryofml.org/terms/decisiontree.html): A decision tree is a flowchart-like representation of a hypothesis map: a root node reads in the feature vector of a data point and evaluates a basic test on it.
- [distribution shift](https://dictionaryofml.org/terms/distshift.html): Distribution shift is a mismatch between the probability distribution underlying the training set and test set of a machine learning method and the probability distribution of the data points that the learned hypothesis faces after deployme
- [eigenvalue](https://dictionaryofml.org/terms/eigenvalue.html): An eigenvalue of a square matrix A is a number λ for which some nonzero vector x satisfies Ax = λx.
- [expectation–maximization](https://dictionaryofml.org/terms/em.html): The expectation–maximization (EM) algorithm is an iterative optimization method for approximately solving maximum likelihood optimization problems that are difficult to solve directly, such as fitting a Gaussian mixture model.
- [ensemble](https://dictionaryofml.org/terms/ensemble.html): An ensemble method combines several machine learning methods, each referred to as a base learner, into one predictor.
- [empirical risk minimization](https://dictionaryofml.org/terms/erm.html): machine learning methods aim to find a hypothesis that yields accurate predictions, reflected by a small loss. empirical risk minimization (ERM) formalizes this: it selects a hypothesis with minimal empirical risk, i.e., minimal average los
- [eigenvalue decomposition](https://dictionaryofml.org/terms/evd.html): An eigenvalue decomposition (EVD) is a factorization of a square matrix of the form A = V Λ V-1.
- [explainability](https://dictionaryofml.org/terms/explainability.html): A machine learning method is explainable if there is an effective way to explain its predictions.
- [explanation](https://dictionaryofml.org/terms/explanation.html): An explanation accompanies a prediction delivered by a machine learning method and says what about the data point drove it.
- [feature](https://dictionaryofml.org/terms/feature.html): A feature of a data point is one of its attributes that can be measured or computed easily, without human supervision.
- [feature importance](https://dictionaryofml.org/terms/featureimportance.html): A feature importance score quantifies, for each feature, how much it contributes to the predictions of a learned hypothesis or to their quality.
- [Gaussian random variable](https://dictionaryofml.org/terms/gaussrv.html): A standard Gaussian random variable (Gaussian RV) is a real-valued random variable with probability density function proportional to (-η2/2).
- [gradient descent](https://dictionaryofml.org/terms/gd.html): Gradient descent (GD) is an iterative algorithm for minimizing a differentiable function f: Rd → R.
- [generalization](https://dictionaryofml.org/terms/generalization.html): Generalization is the ability of a machine learning method to make accurate predictions on data points that have not been used during training.
- [Gaussian mixture model](https://dictionaryofml.org/terms/gmm.html): A Gaussian mixture model (GMM) is a probabilistic model for data points with numeric feature vectors.
- [gradient](https://dictionaryofml.org/terms/gradient.html): The gradient of a real-valued function is a vector that determines the local linear approximation of the function.
- [Hilbert space](https://dictionaryofml.org/terms/hilbertspace.html): A Hilbert space is a complete inner product space: every Cauchy sequence of its elements has a limit in the space.
- [hypothesis space](https://dictionaryofml.org/terms/hypospace.html): A hypothesis space H is a set of hypothesis maps h: X → Y from a feature space into a label space.
- [hypothesis](https://dictionaryofml.org/terms/hypothesis.html): A hypothesis is a map h: X → Y from the feature space X to the label space Y.
- [inner product](https://dictionaryofml.org/terms/innerproduct.html): Many machine learning applications involve data points whose features form numeric arrays.
- [interpretability](https://dictionaryofml.org/terms/interpretability.html): The interpretability of a machine learning method is the extent to which a human user can comprehend the computational process it carries out.
- [interpretable machine learning](https://dictionaryofml.org/terms/interpretableml.html): Interpretable machine learning (interpretable ML) refers to machine learning methods whose hypothesis space contains only hypotheses that a human user can comprehend directly, such as sparse linear models or shallow decision trees.
- [kernel method](https://dictionaryofml.org/terms/kernelmethod.html): A kernel method applies a linear method, such as a linear model or a linear classifier, to transformed feature vectors.
- [k-means](https://dictionaryofml.org/terms/kmeans.html): The k-means method is a hard clustering method for data points with numeric feature vectors.
- [label](https://dictionaryofml.org/terms/label.html): The ultimate goal of machine learning is the accurate prediction of the label of a data point from its features.
- [linear regression](https://dictionaryofml.org/terms/linreg.html): Linear regression is a regression method that learns a linear hypothesis map for predicting the numeric label of a data point from its features.
- [large language model](https://dictionaryofml.org/terms/llm.html): A large language model (LLM) is an artificial neural network, typically with billions of model parameters, that implements a hypothesis map from a prompt to a response.
- [logistic regression](https://dictionaryofml.org/terms/logreg.html): Logistic regression learns a linear hypothesis map h(x) = w x for a binary classification problem with label y ∈ -1, 1.
- [loss](https://dictionaryofml.org/terms/loss.html): The loss incurred by a hypothesis on a single data point is a scalar that quantifies the error of the hypothesis when evaluated on that data point.
- [matrix](https://dictionaryofml.org/terms/matrix.html): A matrix is a rectangular array of numbers arranged in rows and columns, the two-dimensional special case of an array.
- [mean](https://dictionaryofml.org/terms/mean.html): The mean of a random vector x is its expectation, the Lebesgue integral of x with respect to its probability distribution.
- [model](https://dictionaryofml.org/terms/model.html): In machine learning, the word \emph{model} is used in different ways.
- [norm](https://dictionaryofml.org/terms/norm.html): A norm on a vector space is a function that assigns each vector a nonnegative number, its length.
- [overfitting](https://dictionaryofml.org/terms/overfitting.html): Overfitting is the failure mode of a machine learning method that fits its training set too closely: the learned hypothesis incurs a small empirical risk on the training set but a large risk, i.e., the expected loss on data points drawn fro
- [probability distribution](https://dictionaryofml.org/terms/probdist.html): A machine learning method reads in a dataset and delivers an output, such as a learned hypothesis or the prediction for a given data point.
- [projection](https://dictionaryofml.org/terms/projection.html): The projection of a vector onto a closed non-empty subset of a Euclidean space is a point in the subset that is closest to the vector in the Euclidean norm.
- [random forest](https://dictionaryofml.org/terms/randomforest.html): A random forest is an ensemble of decision trees.
- [regularization](https://dictionaryofml.org/terms/regularization.html): Regularization refers to modifications to a machine learning method that improve its generalization.
- [random variable](https://dictionaryofml.org/terms/rv.html): A random variable (RV) is a function that maps each outcome of a random experiment to an element of a measurable space.
- [self-supervised learning](https://dictionaryofml.org/terms/selfsupervisedlearning.html): Self-supervised learning constructs the labels of a training set from the data points themselves.
- [Sobolev space](https://dictionaryofml.org/terms/sobolevspace.html): A Sobolev space consists of the functions whose derivatives, in a generalized sense, have bounded size.
- [stochastic gradient descent](https://dictionaryofml.org/terms/stochGD.html): Stochastic gradient descent (SGD) is a variant of gradient descent in which the gradient of the objective function is replaced by a computationally cheaper stochastic approximation.
- [singular value decomposition](https://dictionaryofml.org/terms/svd.html): The singular value decomposition (SVD) is a factorization of a matrix A ∈ Rm × d of the form A = V Λ U with orthonormal matrices V and U.
- [support vector machine](https://dictionaryofml.org/terms/svm.html): The support vector machine (SVM) is a binary classification method that learns a linear classifier by regularized empirical risk minimization, combining the hinge loss with a squared-norm penalty term.
- [tabular data](https://dictionaryofml.org/terms/tabulardata.html): Tabular data consist of data points that share a common, fixed set of attributes.
- [test set](https://dictionaryofml.org/terms/testset.html): A test set is a dataset of data points used neither for training a model nor for choosing between candidate models based on a validation set.
- [training set](https://dictionaryofml.org/terms/trainset.html): The ultimate goal of machine learning is to learn a hypothesis that accurately predicts the label of any data point based on its features.
- [transformer](https://dictionaryofml.org/terms/transformer.html): Many machine learning applications involve data points that consist of smaller units, so-called tokens, such as the words of a text or the patches of an image.
- [transparency](https://dictionaryofml.org/terms/transparency.html): Transparency is a key requirement for trustworthy artificial intelligence: it names the duties to close the information gaps between the provider of an artificial intelligence system, its deployer, and the persons affected by its prediction
- [validation set](https://dictionaryofml.org/terms/valset.html): A validation set consists of data points which have not been used for model training.
- [variance](https://dictionaryofml.org/terms/variance.html): The variance of a real-valued random variable is the expectation of the squared difference between the random variable and its mean.
- [explainable artificial intelligence](https://dictionaryofml.org/terms/xaiterm.html): Explainable artificial intelligence (XAI) is the subfield of artificial intelligence concerned with making the predictions of machine learning methods understandable to humans.

## Meta

- [List of symbols](https://dictionaryofml.org/symbols.html): the book's notation, one row per symbol
- [How this book is built](https://dictionaryofml.org/how-it-is-made.html): the basic ML picture the entries share, and the editorial pipeline
- [For instructors](https://dictionaryofml.org/for-instructors.html): using the dictionary in a course