Dictionary of Applied Machine Learning
Typeset PDF version — the authoritative form of this entry
A feature of a data point is one of its attributes that can be measured or computed easily, without human supervision. The features of a data point are collected into a feature vector, which a hypothesis map reads to predict the label of the data point. The red-green-blue (RGB) pixel intensities of a digital image or the signal values of an audio recording are typical features. Which attributes of a data point serve as features, rather than as labels, is a design choice within a machine learning (ML) application.
A feature of a data point $\datapoint$ is one of its attributes that is measurable, or can be computed easily, without the need for human supervision (Anon., 2003; Everitt and Skrondal, 2010; Gujarati and Porter, 2009). The features of a data point are assembled into a feature vector $\featurevec = \featuremapvec(\datapoint)$ by a feature map, a function $\featuremapvec: \datapointspace \rightarrow \featurespace$, viewing the data point itself as its raw features from which the computed features are obtained.
For example, if a data point is a digital image, then the
red-green-blue (RGB) intensities of its pixels can serve as features.
Another example is shown in Fig. 1,
where the signal values of a finite-duration audio signal are used
as its features.
Instead of its raw signal values, the audio signal in Fig. 1 can be described by the magnitudes of its discrete-time Fourier transform (DTFT) (Oppenheim et al., 1999). These features capture the frequency content of the signal and are unchanged by time shifts, which is useful for a stationary signal, one whose statistical properties do not change over time. Another example is the activation of a neuron within an artificial neural network (ANN): it is a new feature derived from the input features by a sequence of basic computations encoded by the ANN.
Using activations as features underlies a second, narrower
use of the word feature in mechanistic interpretability:
a feature is a human-interpretable concept encoded inside a trained
ANN, such as a curve detector or the presence of a wheel. The
activations of a subset of $\dimlocalmodel$ neurons
(for example, a whole layer, or the neurons that an
analyst selects for study) form a vector in a
Euclidean space $\reals^{\dimlocalmodel}$. Whereas the broader
sense above treats a single neuron activation as one
feature, here a feature is obtained as a function of this whole
activation vector. One such function is the
projection of the activation vector onto a
one-dimensional subspace spanned by a unit vector (or
direction) (Olah et al., 2020; Park et al., 2024). This
unit vector can be obtained from a hyperplane separating
data points that express the concept from those that do not
(Kim et al., 2018) (see
Fig. 2).
Whether a given attribute is treated as a feature or a label is not inherent to the data point. It is more a design choice that depends on the machine learning (ML) application and the resources available for determining each attribute. An attribute should be used as a feature only if its value can be determined with sufficient accuracy.
The accuracy of features also matters for regulation. The EU AI Act requires the training data of high-risk machine learning systems (ML systems) to be relevant, representative, and as error-free as possible (Parliament and Union, 2024, Art. 10). When features are personal data, the general data protection regulation (GDPR) adds the data minimization principle and the accuracy principle (Parliament and Union, 2016, Art. 5(1)(c) and (d)).
Conversely, deliberately perturbing the features can be useful: adding small perturbations to the features of data points during training is a form of data augmentation that acts as regularization. For example, the ridge regression penalty equals the average loss under Gaussian feature perturbations.
Synonyms: covariate, explanatory variable, independent variable, input, predictor, regressor.
See also: data point, label, feature vector, feature space, dataset, mechanistic interpretability, concept activation vector, EU AI Act, data augmentation, regularization, ridge regression, Fourier transform.
@misc{dictml_feature,
author = {Jung, Alexander},
title = {feature},
howpublished = {Dictionary of Applied Machine Learning (course edition)},
year = {2026},
doi = {10.5281/zenodo.21569296},
note = {ISBN 978-952-64-3013-3, CC BY 4.0, retrieved 2026-08-06},
url = {https://dictionaryofml.org/terms/feature.html}
}