# feature vector *Entry of the [Dictionary of Applied Machine Learning](https://dictionaryofml.org) (course edition), by Alexander Jung, Aalto University. CC BY 4.0. Updated 2026-11-08. [HTML](https://dictionaryofml.org/terms/featurevec.html) · [typeset PDF](https://dictionaryofml.org/terms/featurevec.pdf) · [LaTeX](https://dictionaryofml.org/terms/featurevec.tex) · [Python demo](https://dictionaryofml.org/terms/featurevec.py) · [notebook](https://dictionaryofml.org/terms/featurevec.ipynb)* Math is LaTeX with the book's macros expanded, so no notation package is needed to read it. ## Abstract A feature vector is a list of numbers describing a data point, with one entry for each of its features. A feature is an attribute of the data point that is measured or computed without human supervision, so the length of the vector is the number of such attributes. A feature transformation computes the feature vector from the data point and delivers a point of the feature space, to which a hypothesis map is then applied to predict the label. Non-numeric features enter as numbers: a binary attribute as $0$ or $1$, a categorical one through one-hot encoding. Most machine learning methods use feature vectors in a Euclidean space whose dimension is the number of features. A feature vector can more generally be an element of a Hilbert space, which is how a kernel method uses one without ever forming it, using it only through inner products. ## Definition A house offered for sale can be described to a machine learning method by three numbers: $85$ square meters of living area, $3$ rooms, and $1974$ as the year of construction. That list $\big(85, 3, 1974\big)$ is the feature vector of the house. In general, a feature vector gathers into one vector the features of a data point, those attributes of the data point that are measured or computed without human supervision (Duda et al., 2001). A list of $d$ real numbers, for a natural number $d$, is an element of the Euclidean space $\mathbb{R}^{d}$ (Strang, 2016, Sect. 1.1). The feature vector of a data point ${\mathbf z}$ is written ${\mathbf x} = \big(x_{1}, \,\ldots, \,x_{d}\big)^{\top}$, and its entry $x_{j}$ is one feature of that data point. It is obtained from the data point by a feature transformation ${\mathbf \Phi} : \mathcal{Z} \rightarrow \mathcal{X}$, ${\mathbf x} = {\mathbf \Phi}({\mathbf z})$, which views the data point itself as its raw features and delivers a point of the feature space $\mathcal{X}$ (Fig. 1). A hypothesis map takes the feature vector as input and outputs a prediction of the label of the data point. ![Figure 1 of the entry featurevec](https://dictionaryofml.org/terms/fig/featurevec-fig1.svg) *Figure 1: A feature vector ${\mathbf x}$ stacks the individual features $x_{1}, \,\ldots, \,x_{d}$ of a data point ${\mathbf z}$, computed by the feature transformation ${\mathbf \Phi}$. It is a point in the feature space $\mathcal{X}$* Non-numeric features can also sit in the entries of a feature vector, once their values are encoded as real numbers (Hastie et al., 2009; Goodfellow et al., 2016). A binary feature, such as whether the house has a balcony, becomes $0$ or $1$. A categorical feature with $K$ possible values, such as the heating type (gas, oil, or electric), is one-hot encoded: $K$ entries of the feature vector carry the value $0$ except for a single entry equal to $1$, whose position names the value. A feature transformation is also defined for data points that are not records of numbers. A text becomes a feature vector by counting how often each entry of a fixed word list appears in it, which gives one entry per listed word. A digital image is an array of pixels, and the red, green and blue intensities of those pixels become one entry each. Reading a $2 \times 2$ patch of such an image this way gives $2 \cdot 2 \cdot 3 = 12$ features (Fig. 2). An audio recording becomes a feature vector through the values its signal takes at successive instants, one entry $x_{j}$ per instant $j$. The feature transformation differs in each case, and so does the dimension $d$ of the resulting feature space: the house above needs seven entries once its balcony and heating type are encoded, while an audio recording read at $256$ instants needs $256$ (Fig. 3). The order of the entries is arbitrary but fixed: every data point must be read in the same order, since a hypothesis map weights the entry at a given position, not the feature at that position in another data point. ![Figure 2 of the entry featurevec](https://dictionaryofml.org/terms/fig/featurevec-fig2.svg) *Figure 2: A $2 \times 2$ patch of a digital image read as a feature vector. The patch carries one array of intensities per color channel, drawn as the three grids labeled R, G and B; each grid holds the same four pixels in the same positions. On the right the patch is shown in color, its pixels numbered $1$ to $4$: pixel $1$ is the color that the three intensities $78$, $85$ and $67$ make together. Stacking those three values, then those of pixel $2$, and so on gives a feature vector with $d = 12$ entries* ![Figure 3 of the entry featurevec](https://dictionaryofml.org/terms/fig/featurevec-fig3.svg) *Figure 3: The feature vector of an audio recording read at $256$ instants. Entry $x_{j}$ is the value its signal takes at instant $j$, so this data point is a point of $\mathbb{R}^{256}$. Data generated by pythondemos/featurevec.py* Most machine learning methods use feature vectors in a finite-dimensional Euclidean space $\mathbb{R}^{d}$, whose dimension $d$ is the number of features. More generally, a feature vector can be an element of an abstract Hilbert space $\mathcal{H}$. One way to build a feature vector of that kind starts from a kernel $k: \mathcal{X} \times \mathcal{X} \rightarrow \mathbb{R}$ on a given feature space $\mathcal{X}$. The kernel assigns to each ${\mathbf x} \in \mathcal{X}$ the function $k\big({\mathbf x},\cdot\big): \mathcal{X} \rightarrow \mathbb{R}$, whose value at ${\mathbf x}' \in \mathcal{X}$ is $k\big({\mathbf x},{\mathbf x}'\big)$. That function is itself a vector in a reproducing kernel Hilbert space $\mathcal{H}$, a Hilbert space whose elements are functions on $\mathcal{X}$ (Aronszajn, 1950, Part I, Sect. 2; Schölkopf and Smola, 2002, Sect. 2.2.2). A kernel method uses these feature vectors without ever forming them explicitly: it reaches them only through their inner products, which the reproducing property reduces to kernel evaluations, $\langle k\big({\mathbf x},\cdot\big), k\big({\mathbf x}',\cdot\big) \rangle_{\mathcal{H}} = k\big({\mathbf x},{\mathbf x}'\big)$ (Schölkopf and Smola, 2002, Eq. (2.30)). **Synonyms:** covariate vector, input vector, predictor vector. **See also:** [feature](https://dictionaryofml.org/terms/feature.html), feature space, feature transformation, [data point](https://dictionaryofml.org/terms/datapoint.html), vector, Euclidean space, kernel, reproducing kernel Hilbert space, [kernel method](https://dictionaryofml.org/terms/kernelmethod.html), [Hilbert space](https://dictionaryofml.org/terms/hilbertspace.html). ## References - Duda et al. (2001). Pattern Classification. *Wiley*. - Strang (2016). Introduction to Linear Algebra. *Wellesley-Cambridge Press*. - Hastie et al. (2009). The Elements of Statistical Learning: Data Mining, Inference, and Prediction. *Springer Science+Business Media*. - Goodfellow et al. (2016). Deep Learning. *MIT Press*. - Aronszajn (1950). Theory of Reproducing Kernels. *Trans. Am. Math. Soc.*. - Schölkopf and Smola (2002). Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. *MIT Press*. ## Cite this entry ```bibtex @misc{dictml_featurevec, author = {Jung, Alexander}, editor = {Olioumtsevits, Konstantina and Schnoor, Ekkehard}, title = {feature vector}, howpublished = {Dictionary of Applied Machine Learning (course edition)}, year = {2026}, doi = {10.5281/zenodo.21569296}, note = {ISBN 978-952-64-3013-3, CC BY 4.0, retrieved 2026-11-11}, url = {https://dictionaryofml.org/terms/featurevec.html} } ```