Dictionary of Applied Machine Learning: 107 machine-learning terms, each with a definition, a figure, citations and a Python demo · All terms · Demos · Start here

accuracy

Updated on 2026-10-21

Readiness 100/100

▶ Run the Python demo Open in Colab Typeset PDF LaTeX source Cite this entry

See also loss classification confusion matrix

Accuracy is the fraction of correct predictions made by a hypothesis on a dataset with a finite label space, equal to one minus the average $0/1$ loss. It is widely used as a single-number metric for classification methods. Accuracy is not suitable as an objective function for training, since the $0/1$ loss is non-smooth and non-convex. Empirical risk minimization (ERM) uses a surrogate loss instead, a convex or smooth upper bound on the $0/1$ loss, and accuracy is reported on a validation set or a test set. On imbalanced data, accuracy alone cannot tell a classifier from a baseline that ignores the features, which calls for other means of evaluation such as the confusion matrix.

Figure produced by the Python demo of the entry accuracy
Figure written by the entry's Python demo accuracy.py, which recomputes what the entry states: the script block by block · open in Colab

Definition

B-dataAccuracy is the fraction of correct predictions made by a hypothesis $\hypothesis: \featurespace \rightarrow \labelspace$ on a dataset $\dataset = \big\{ \big(\featurevec^{(\sampleidx)}, \truelabel^{(\sampleidx)} \big) \big\}_{\sampleidx=1}^{\samplesize}$ with a finite label space $\labelspace$ (Goodfellow et al., 2016, Sect. 5.1.2; Shalev-Shwartz and Ben-David, 2014, Sect. 2.2). With the $0/1$ loss $\lossfunczo{\datapoint}{\hypothesis}$, which is $1$ for a wrong prediction on the data point $\datapoint$ and $0$ otherwise, accuracy is \begin{align*} \operatorname{acc}(\hypothesis \mid \dataset) & \defeq 1 - \frac{1}{\samplesize} \sum_{\sampleidx=1}^{\samplesize} \lossfunczo{\big(\featurevec^{(\sampleidx)}, \truelabel^{(\sampleidx)}\big)}{\hypothesis} \\ & = \frac{1}{\samplesize} \Big| \Big\{ \sampleidx \in \{1, \,\ldots, \,\samplesize\} : \hypothesis\big(\featurevec^{(\sampleidx)}\big) = \truelabel^{(\sampleidx)} \Big\} \Big| \text{.} \end{align*} Accuracy ranges from $0$ (no prediction is correct) to $1$ (every prediction is correct). Equivalently, accuracy equals one minus the average $0/1$ loss on the dataset. For example, in image classification with the label space $\labelspace = \{\text{cat}, \text{dog}, \text{bird}\}$, the hypothesis assigns each image, represented by its feature vector $\featurevec$, a category $\hypothesis(\featurevec)$. Accuracy is then the fraction of images whose assigned category equals the true label $\truelabel$: $94$ correct assignments among $100$ images give an accuracy of $0.94$.

B-linearFig. 1 compares two classifiers on the same training set $\trainset$ of $60$ data points, the dataset from which both were learned. The linear classifier $\hypothesis$, whose decision boundary is a line, misclassifies three data points near that line. The nonlinear classifier $\hypothesis'$ curves around each of these data points and classifies them correctly.

Figure 1 of the entry accuracy
Figure 1: Two classifiers applied to the same training set $\trainset$ of $\samplesize=60$ data points with classes $\truelabel \in \{0,1\}$ (blue circles and red squares). The linear classifier $\hypothesis$ (dashed line) places three data points on the wrong side of its decision boundary; they are ringed in black and lie close to the dashed line, and they yield $\operatorname{acc}(\hypothesis \mid \trainset) = 57/60 = 0.95$. The nonlinear classifier $\hypothesis'$ (solid green curve) bends around those three data points and yields $\operatorname{acc}(\hypothesis' \mid \trainset) = 1$. Data generated by pythondemos/accuracy.py
The classifier $\hypothesis'$ in Fig. 1 achieves the optimal accuracy of $1$ on $\trainset$: its decision boundary closely follows the positions of the individual data points that the linear classifier misclassifies. The classifier $\hypothesis'$ is therefore sensitive to small perturbations of the feature vectors, such as measurement noise. A slight shift in the feature vector of a data point near the decision boundary can flip the prediction. Optimizing accuracy on the training set therefore does not by itself ensure good generalization. Good generalization means a small loss on data points outside the training set, such as the temperatures of days that did not belong to it. Accuracy is often used as a single-number metric to compare learned classifiers: on a validation set $\valset$, to select among candidate hypotheses during validation, and on a test set $\testset$, whose data points entered neither training nor selection, to assess the selected hypothesis. Accuracy is the metric of choice when the classes are balanced and every error costs the same; on imbalanced data it is not.

Accuracy could also be used as the objective function for training. However, accuracy is defined via the $0/1$ loss, which is piecewise constant as a function of the model parameters, so accuracy is piecewise constant too: the gradient of accuracy is zero wherever it exists and therefore indicates no direction of improvement. Gradient-based optimization methods cannot maximize accuracy directly. Instead, empirical risk minimization (ERM)-based methods use a surrogate loss that upper bounds the $0/1$ loss and is chosen convex or smooth, so that those methods have a gradient to follow (Shalev-Shwartz and Ben-David, 2014, Sect. 12.3). Two examples for such a surrogate loss are the logistic loss, which is convex and smooth, and the hinge loss, which is convex but non-smooth: not differentiable at margin $1$.

Fig. 2 shows the three loss functions as functions of the margin, for binary classification with $\labelspace = \{-1, 1\}$, where a real-valued hypothesis $\hypothesis$ predicts via the sign of $\hypothesis(\featurevec)$. Each loss function depends on the data point only through the margin $\truelabel \cdot \hypothesis(\featurevec)$, which is how loss functions for binary classification are compared (Hastie et al., 2009, Sect. 10.6). The hinge loss lies above the $0/1$ loss everywhere; the logistic loss equals $\ln 2$ at margin $0$, so it is the logistic loss scaled by $1 / \ln 2$ that lies above the $0/1$ loss.

Figure 2 of the entry accuracy
Figure 2: Three loss functions for binary classification as functions of the margin $\truelabel \cdot \hypothesis(\featurevec)$: the $0/1$ loss (blue step; the filled dot marks its value $1$ at margin $0$, the open dot the value it does not take), the hinge loss (red curve), and the logistic loss scaled by $1/\ln 2$ (green curve). All three pass through $1$ at margin $0$. Both surrogates are convex, lie above the $0/1$ loss, and penalize small margins where the $0/1$ loss is already $0$
Accuracy can also be misleading for imbalanced data. This failure is a property of accuracy as a metric, so it arises in training and validation alike. Consider a frost warning for Krems an der Donau (Austria): the features of a data point are the minimum and the maximum air temperature of one day, and its label indicates whether the minimum temperature of the following day falls below $0\,^{\circ}\mathrm{C}$. Frost days are the minority class: of a training set of $40$ recorded days, only $6$ are followed by frost.

B-overfitThe constant prediction "above zero" (a baseline that ignores the features) is therefore correct on $34$ of the $40$ days, an accuracy of $0.85$, and a classifier learned by logistic regression from the $40$ data points reaches the same accuracy of $0.85$. Both make $6$ errors and $34$ correct predictions on the $40$ days, so accuracy cannot tell the two apart; they differ in where they are right, which calls for other means of evaluation such as the confusion matrix. The confusion matrices in Fig. 3 show that the learned classifier detects one of the six frost days and raises one false alarm, while the baseline detects none and raises none.

Figure 3 of the entry accuracy
Figure 3: The confusion matrices of the frost-warning classifier learned by logistic regression and of the always-above-zero baseline. The shaded diagonal cells hold the correct predictions: $33 + 1 = 34$ for the learned classifier and $34 + 0 = 34$ for the baseline, so both reach the same accuracy of $0.85$ with $6$ errors each. The cells differ in where the correct and the wrong predictions fall: the learned classifier detects one frost day and raises one false alarm, while the baseline detects none and raises none. Data generated by pythondemos/cm.py

See also: $\bf 0/1$ loss, loss, classification, F$_1$ score, confusion matrix, imbalanced data.

References

  1. Goodfellow et al. (2016). Deep Learning. MIT Press.
  2. Shalev-Shwartz and Ben-David (2014). Understanding Machine Learning: From Theory to Algorithms. Cambridge Univ. Press. doi.org/10.1017/cbo9781107298019
  3. Hastie et al. (2009). The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer Science+Business Media. doi.org/10.1007/978-0-387-84858-7

Cite this entry

@misc{dictml_accuracy,
  author = {Jung, Alexander},
  editor = {Olioumtsevits, Konstantina and Schnoor, Ekkehard},
  title = {accuracy},
  howpublished = {Dictionary of Applied Machine Learning (course edition)},
  year = {2026},
  doi = {10.5281/zenodo.21569296},
  note = {ISBN 978-952-64-3013-3, CC BY 4.0, retrieved 2026-10-27},
  url = {https://dictionaryofml.org/terms/accuracy.html}
}