feature.py, which recomputes what the entry states: the script block by block · open in ColabDictionary of Applied Machine Learning: 114 machine-learning terms, each with a definition, a figure, citations and a Python demo · All terms · Demos · Start here
Updated on 2026-10-29
Readiness 100/100
See also data point label feature vector concept activation vector data augmentation
A feature of a data point is one of its attributes that can be measured or computed easily, without human supervision. The features of a data point are collected into a feature vector, which a hypothesis map reads to predict the label of the data point. The red-green-blue (RGB) pixel intensities of a digital image or the signal values of an audio recording are typical features. Which attributes of a data point serve as features, rather than as labels, is a design choice within a machine learning (ML) application.
feature.py, which recomputes what the entry states: the script block by block · open in ColabP-featmapA digital photo of a cow is a grid of pixels, and the red, green and blue intensity of each pixel is a number that a computer obtains without any human looking at the photo. Such a number is a feature of the data point: an attribute of the data point ${\mathbf z}$ that is measured, or computed, without human supervision (Anon., 2003; Everitt and Skrondal, 2010; Gujarati and Porter, 2009). The features of a data point are assembled into a feature vector ${\mathbf x} = {\mathbf \Phi}({\mathbf z})$ by a feature transformation, a function ${\mathbf \Phi}: \mathcal{Z} \rightarrow \mathcal{X}$, viewing the data point itself as its raw features from which the computed features are obtained. A hypothesis map takes the feature vector as input and outputs a prediction of the label of the data point.
Besides the red-green-blue (RGB) intensities of the pixels of
a photo, Fig. 1 shows another
example of features: the $d$ signal values of a
finite-duration audio signal, used as its features
$x_{1}, \,\ldots, \,x_{d}$.
P-transformOnce the features are chosen, new features can be constructed by transforming existing ones. Such a transformation can be an arithmetic computation applied to the existing features. For example, if a data point has a numeric feature $x \in \mathbb{R}$, the quantities $x^{2}$, $\exp(-x)$, and $3x$ can also be used as features of the data point.
P-activationInstead of its raw signal values, the audio signal in Fig. 1 can be described by the magnitudes of its discrete-time Fourier transform (DTFT) (Oppenheim et al., 1999). These features describe the frequency content of the signal and are unchanged by time shifts, which is useful for a stationary signal, one whose statistical properties do not change over time. Another example is the activation of a neuron within an artificial neural network (ANN): it is a new feature derived from the input features by a sequence of basic computations performed by the ANN.
P-mechfeatUsing activations as features underlies a second, narrower
use of the word feature in mechanistic interpretability:
a feature is a human-interpretable concept encoded inside a trained
ANN, such as a curve detector or the presence of a wheel. The
activations of a subset of $d$ neurons
(for example, a whole layer, or the neurons that an
analyst selects for study) form a vector in a
Euclidean space $\mathbb{R}^{d}$. Whereas the broader
sense above treats a single neuron activation as one
feature, here a feature is obtained as a function of this whole
activation vector ${\mathbf a}$. One such function is the
projection of the activation vector ${\mathbf a}$ onto a
one-dimensional subspace spanned by a unit vector (or
direction) ${\mathbf u}$ (Olah et al., 2020; Park et al., 2024). This
unit vector can be obtained as the unit normal ${\mathbf u}$ of a
hyperplane separating
data points that express the concept from those that do not
(Kim et al., 2018) (see
Fig. 2).
Beyond their construction, regulation constrains which features may be used and how accurate they must be. The EU AI Act requires the training data of high-risk machine learning systems (ML systems) to be relevant, sufficiently representative, and, to the best extent possible, free of errors and complete (European Parliament and Council of the European Union, 2024, Art. 10(3)). When features are personal data, the general data protection regulation (GDPR) adds the data minimization principle (European Parliament and Council of the European Union, 2016, Art. 5(1)(c)) and the accuracy principle (European Parliament and Council of the European Union, 2016, Art. 5(1)(d)).
Conversely, deliberately perturbing the features can be useful: adding small perturbations to the features of data points during training is a form of data augmentation that acts as regularization. For example, the ridge regression penalty equals the average loss under Gaussian feature perturbations: for a linear model with model parameters ${\mathbf w}$, adding zero-mean Gaussian noise of variance $\alpha$ to each feature raises the expected squared error loss by $\alpha \left\lVert {\mathbf w} \right\rVert_{2}^{2}$, the ridge penalty (Bishop, 1995).
Synonyms: covariate, explanatory variable, independent variable, input, predictor, regressor.
See also: data point, label, feature vector, feature space, mechanistic interpretability, concept activation vector, data augmentation.
Published version: doi.org/10.1007/978-981-92-2012-0_11-1, in Dictionary of Applied Machine Learning (Springer Nature Singapore). Please cite that version. This page corresponds to version 1 of that entry.
@incollection{dictml_feature,
author = {Jung, Alexander},
editor = {Olioumtsevits, Konstantina and Schnoor, Ekkehard},
title = {feature},
booktitle = {Dictionary of Applied Machine Learning},
publisher = {Springer Nature Singapore},
year = {2026},
doi = {10.1007/978-981-92-2012-0_11-1},
isbn = {978-981-92-2012-0},
url = {https://doi.org/10.1007/978-981-92-2012-0_11-1}
}