# feature *Entry of the [Dictionary of Applied Machine Learning](https://dictionaryofml.org) (course edition), by Alexander Jung, Aalto University. CC BY 4.0. Updated 2026-10-29. [HTML](https://dictionaryofml.org/terms/feature.html) · [typeset PDF](https://dictionaryofml.org/terms/feature.pdf) · [LaTeX](https://dictionaryofml.org/terms/feature.tex) · [Python demo](https://dictionaryofml.org/terms/feature.py) · [notebook](https://dictionaryofml.org/terms/feature.ipynb)* Math is LaTeX with the book's macros expanded, so no notation package is needed to read it. ## Abstract A feature of a data point is one of its attributes that can be measured or computed easily, without human supervision. The features of a data point are collected into a feature vector, which a hypothesis map reads to predict the label of the data point. The red-green-blue (RGB) pixel intensities of a digital image or the signal values of an audio recording are typical features. Which attributes of a data point serve as features, rather than as labels, is a design choice within a machine learning application. ## Definition A digital photo of a cow is a grid of pixels, and the red, green and blue intensity of each pixel is a number that a computer obtains without any human looking at the photo. Such a number is a feature of the data point: an attribute of the data point ${\mathbf z}$ that is measured, or computed, without human supervision (Anon., 2003; Everitt and Skrondal, 2010; Gujarati and Porter, 2009). The features of a data point are assembled into a feature vector ${\mathbf x} = {\mathbf \Phi}({\mathbf z})$ by a feature transformation, a function ${\mathbf \Phi}: \mathcal{Z} \rightarrow \mathcal{X}$, viewing the data point itself as its raw features from which the computed features are obtained. A hypothesis map takes the feature vector as input and outputs a prediction of the label of the data point. Besides the red-green-blue (RGB) intensities of the pixels of a photo, Fig. 1 shows another example of features: the $d$ signal values of a finite-duration audio signal, used as its features $x_{1}, \,\ldots, \,x_{d}$. ![Figure 1 of the entry feature](https://dictionaryofml.org/terms/fig/feature-fig1.svg) *Figure 1: Audio signal (curve) against time and its signal values at the sampling instants (dots), which can be used as features $x_{1},\,\ldots,\,x_{d}$* Whether a given attribute is treated as a feature or a label is not inherent to the data point. It is a design choice that depends on the machine learning application and the resources available for determining each attribute. An attribute should be used as a feature only if its value can be determined for every data point at the time the prediction is needed; an attribute that becomes known only afterwards is a candidate for the label instead. Once the features are chosen, new features can be constructed by transforming existing ones. Such a transformation can be an arithmetic computation applied to the existing features. For example, if a data point has a numeric feature $x \in \mathbb{R}$, the quantities $x^{2}$, $\exp(-x)$, and $3x$ can also be used as features of the data point. Instead of its raw signal values, the audio signal in Fig. 1 can be described by the magnitudes of its discrete-time Fourier transform (DTFT) (Oppenheim et al., 1999). These features describe the frequency content of the signal and are unchanged by time shifts, which is useful for a stationary signal, one whose statistical properties do not change over time. Another example is the activation of a neuron within an artificial neural network: it is a new feature derived from the input features by a sequence of basic computations performed by the artificial neural network. Using activations as features underlies a second, narrower use of the word *feature* in mechanistic interpretability: a feature is a human-interpretable concept encoded inside a trained artificial neural network, such as a curve detector or the presence of a wheel. The activations of a subset of $d$ neurons (for example, a whole layer, or the neurons that an analyst selects for study) form a vector in a Euclidean space $\mathbb{R}^{d}$. Whereas the broader sense above treats a single neuron activation as one feature, here a feature is obtained as a function of this whole activation vector ${\mathbf a}$. One such function is the projection of the activation vector ${\mathbf a}$ onto a one-dimensional subspace spanned by a unit vector (or direction) ${\mathbf u}$ (Olah et al., 2020; Park et al., 2024). This unit vector can be obtained as the unit normal ${\mathbf u}$ of a hyperplane separating data points that express the concept from those that do not (Kim et al., 2018) (see Fig. 2). ![Figure 2 of the entry feature](https://dictionaryofml.org/terms/fig/feature-fig2.svg) *Figure 2: A feature as a function of an activation vector ${\mathbf a}$ obtained from a trained artificial neural network. (a) An artificial neural network with one hidden layer of three neurons; the activations $a_{1}, a_{2}$ of a selected subset (dashed box) form the activation vector ${\mathbf a}=\big(a_{1},a_{2}\big)^{\top}$, while $a_{3}$ is not used. (b) Each data point maps to a point $(a_{1}, a_{2})^{\top}$ in the activation space. Data points that express the concept (filled circles) and those that do not (open squares) are separated by a hyperplane (dashed line) with unit normal ${\mathbf u}$ (thick arrow). The feature value is the projection of the activation vector onto the line spanned by ${\mathbf u}$ (thin solid line), shown for two points by dotted lines ending in small dots* Another construction of an activation-based feature is based on regions instead of directions in the activation space (Crabbé and van der Schaar, 2022). In either case, constructing the function that maps the activation vector to the feature requires a set of data points labeled as representing the concept or not. Beyond their construction, regulation constrains which features may be used and how accurate they must be. The EU AI Act requires the training data of high-risk ML systems to be relevant, sufficiently representative, and, to the best extent possible, free of errors and complete (European Parliament and Council of the European Union, 2024, Art. 10(3)). When features are personal data, the general data protection regulation adds the data minimization principle (European Parliament and Council of the European Union, 2016, Art. 5(1)(c)) and the accuracy principle (European Parliament and Council of the European Union, 2016, Art. 5(1)(d)). Conversely, deliberately perturbing the features can be useful: adding small perturbations to the features of data points during training is a form of data augmentation that acts as regularization. For example, the ridge regression penalty equals the average loss under Gaussian feature perturbations: for a linear model with model parameters ${\mathbf w}$, adding zero-mean Gaussian noise of variance $\alpha$ to each feature raises the expected squared error loss by $\alpha \left\lVert {\mathbf w} \right\rVert_{2}^{2}$, the ridge penalty (Bishop, 1995). **Synonyms:** covariate, explanatory variable, independent variable, input, predictor, regressor. **See also:** [data point](https://dictionaryofml.org/terms/datapoint.html), [label](https://dictionaryofml.org/terms/label.html), [feature vector](https://dictionaryofml.org/terms/featurevec.html), feature space, mechanistic interpretability, [concept activation vector](https://dictionaryofml.org/terms/cav.html), [data augmentation](https://dictionaryofml.org/terms/dataaug.html). ## References - Anon. (2003). The Oxford Dictionary of Statistical Terms. *Oxford Univ. Press*. - Everitt and Skrondal (2010). The Cambridge Dictionary of Statistics. *Cambridge Univ. Press*. - Gujarati and Porter (2009). Basic Econometrics. *McGraw-Hill/Irwin*. - Oppenheim et al. (1999). Discrete-Time Signal Processing. *Prentice Hall*. - Olah et al. (2020). Zoom In: An Introduction to Circuits. *Distill*. - Park et al. (2024). The Linear Representation Hypothesis and the Geometry of Large Language Models. *Proc. 41st Int. Conf. Mach. Learn.*. - Kim et al. (2018). Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (TCAV). *Proc. 35th Int. Conf. Mach. Learn.*. - Crabbé and van der Schaar (2022). Concept Activation Regions: A Generalized Framework for Concept-Based Explanations. *Adv. Neural Inf. Process. Syst.*. - European Parliament and Council of the European Union (2024). Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 laying down harmonised rules on artificial intelligence and amending Regulations (EC) No 300/2008, (EU) No 167/2013, (EU) No 168/2013, (EU) 2018/858, (EU) 2018/1139 and (EU) 2019/2144 and Directives 2014/90/EU, (EU) 2016/797 and (EU) 2020/1828 (Artificial Intelligence Act) (Text with EEA relevance). - European Parliament and Council of the European Union (2016). Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation) (Text with EEA relevance). - Bishop (1995). Training with Noise is Equivalent to Tikhonov Regularization. *Neural Computation*. ## Cite this entry ```bibtex @incollection{dictml_feature, author = {Jung, Alexander}, editor = {Olioumtsevits, Konstantina and Schnoor, Ekkehard}, title = {feature}, booktitle = {Dictionary of Applied Machine Learning}, publisher = {Springer Nature Singapore}, year = {2026}, doi = {10.1007/978-981-92-2012-0_11-1}, isbn = {978-981-92-2012-0}, url = {https://doi.org/10.1007/978-981-92-2012-0_11-1} } ```