# artificial neural network (ANN)

*Entry of the [Dictionary of Applied Machine Learning](https://dictionaryofml.org) (course edition), by Alexander Jung, Aalto University. CC BY 4.0. Updated 2026-11-08. [HTML](https://dictionaryofml.org/terms/ann.html) · [typeset PDF](https://dictionaryofml.org/terms/ann.pdf) · [LaTeX](https://dictionaryofml.org/terms/ann.tex) · [Python demo](https://dictionaryofml.org/terms/ann.py) · [notebook](https://dictionaryofml.org/terms/ann.ipynb)*

Math is LaTeX with the book's macros expanded, so no notation package is needed to read it.

## Abstract

An artificial neural network (ANN) is a hypothesis space whose hypothesis maps are all built on one directed graph. Each hypothesis in this space is a map from the features of a data point to a prediction for the label, obtained for a particular choice of the learnable model parameters carried by the graph's edges (edge weights) and nodes (offset terms). Each node is an artificial neuron that applies an activation function to the weighted sum of its inputs plus its offset. The connectivity pattern, the architecture, is an important design choice: a feedforward network uses a directed acyclic graph, whereas a recurrent neural network allows a cycle. Depending on the connectivity, ANNs include deep nets, CNNs, RNNs, and transformer-based architectures.

## Definition

The face recognition of a smartphone camera and the speech transcription of a voice assistant are computed by ANNs whose connection strengths were learned from data (Goodfellow et al., 2016, Sects. 12.2 and 12.3). An ANN is a hypothesis space whose hypothesis maps are all built on one directed graph: its nodes are the artificial neurons, elementary computational units, and its weighted edges fix which artificial neurons feed which. The edge weights and the offset terms assigned to the nodes form the learnable model parameters ${\mathbf w}$ of an ANN, and a choice of ${\mathbf w}$ selects one hypothesis map $h^{({\mathbf w})}$, from the features of a data point at the input to an output used as the prediction for the label. The learned hypothesis $\widehat{h} = h^{(\widehat{{\mathbf w}})}$ is one such choice.

Each artificial neuron applies an activation function $\sigma(\cdot)$ to the weighted sum of its inputs plus an offset term (Fig. 1). The activation function can itself carry model parameters (parametric activation functions, He et al., 2015), but it is customarily prescribed in advance, for example the rectified linear unit (Goodfellow et al., 2016, Sect. 6.3), so that only the node-wise offset and the edge weights are learned.

![Figure 1 of the entry ann](https://dictionaryofml.org/terms/fig/ann-fig1.svg)

*Figure 1: A single artificial neuron: it applies an activation function $\sigma(\cdot)$ to the weighted sum $w_{1} x_1 + w_{2} x_2$ of its inputs plus an offset term $b$, and delivers the output $a_{1}$. The edge weights $w_{1}$, $w_{2}$ and the offset $b$ are learnable model parameters*

The output of a neuron can be used either as the final output of the ANN or as an input to other neurons. The directed graph of the ANN records these uses: its nodes are the artificial neurons together with the input features, and each directed edge carries the output of one node to the input of another (Fig. 2). Which outputs are connected to which downstream neuron's inputs, the connectivity structure (or architecture) of the ANN, is therefore an important design aspect (Goodfellow et al., 2016). Whether the graph is acyclic or contains a cycle separates the two broad families of ANN.

![Figure 2 of the entry ann](https://dictionaryofml.org/terms/fig/ann-fig2.svg)

*Figure 2: An ANN represented as a weighted, acyclic directed graph with nodes corresponding to neurons or features of a data point. Feature nodes $x_1$, $x_2$ have a fixed output given by the feature value, with no input. Each of the neuron nodes $a_{1}$, $a_{2}$ applies an activation function to the weighted sum of its inputs (each as in Fig. 1, whose offset terms are omitted here). The weighted directed edges, with arrowheads marking the direction of signal flow, indicate how node outputs are used as inputs to downstream neurons. The edge weights are tunable model parameters and are used to scale the inputs to the neurons. The prediction $h^{({\mathbf w})}({\mathbf x})$ is the final output of the ANN, computed from the inputs $x_1$, $a_1$, and $a_2$*

The graph of Fig. 2 is acyclic, so this ANN is feedforward (Goodfellow et al., 2016, Ch. 6): the signal passes from the inputs to the output without returning to any node. One widely used type of ANN is deep nets where neurons form consecutive layers (Fig. 3). In a deep net, the outputs of neurons in a given layer are typically only connected to the inputs of the neurons in a consecutive layer. By contrast, in the general directed acyclic graph of Fig. 2 an output may feed any downstream neuron, and there $x_1$ connects straight to the prediction.

Each layer $l$ then applies a feature transformation ${\mathbf \Phi}^{(l)}$ whose components are computed by the neurons of that layer (each as in Fig. 1), and the deep net computes the concatenation of these layer-wise maps. A shortcut or skip connection directly connects the outputs of neurons in one layer to the inputs of neurons in a nonconsecutive layer (Goodfellow et al., 2016; He et al., 2016); Fig. 3 shows one as a dashed edge. With that edge removed, the network of Fig. 3 delivers the prediction

$$
h^{({\mathbf w})}({\mathbf x})
= {\mathbf \Phi}^{(3)}\Big({\mathbf \Phi}^{(2)}
\big({\mathbf \Phi}^{(1)}({\mathbf x})\big)\Big)
\text{.}
$$

Here, the input is the feature vector ${\mathbf x} = (x_1, x_2, x_3)^{\top}$. The two hidden layers compute ${\mathbf \Phi}^{(1)}, {\mathbf \Phi}^{(2)}:
\mathbb{R}^{3} \rightarrow \mathbb{R}^{3}$, and the output layer computes ${\mathbf \Phi}^{(3)}: \mathbb{R}^{3} \rightarrow
\mathbb{R}$.

![Figure 3 of the entry ann](https://dictionaryofml.org/terms/fig/ann-fig3.svg)

*Figure 3: A deep net: the neurons form consecutive layers, and each neuron receives only the outputs of the preceding layer. Every solid edge carries a learnable edge weight (not drawn). The dashed edge is a skip connection, directly connecting two nonconsecutive layers*

The graphs seen so far are acyclic. A recurrent neural network instead includes a feedback edge (Goodfellow et al., 2016, Ch. 10), so its graph contains a cycle and is not a directed acyclic graph (Fig. 4): a self-loop feeds a neuron's output back to its own input. This feedback lets the network carry information across the steps of a sequence; unrolling the recurrence over the sequence, one copy of each neuron per step, recovers a directed acyclic graph (Goodfellow et al., 2016, Sect. 10.1).

![Figure 4 of the entry ann](https://dictionaryofml.org/terms/fig/ann-fig4.svg)

*Figure 4: A simple recurrent neural network. The self-loop on the hidden neuron $h$ (edge weight $w_{\mathrm{rec}}$) feeds its output back to its own input, so the graph contains a cycle and is not a directed acyclic graph. The feedback carries information across the steps of a sequence*

The networks of Figs. 2, 3, and 4 differ only in which nodes their directed edges join. This connectivity pattern names the common architectures. Feedforward networks have an acyclic graph, a directed acyclic graph (deep nets are the layered case), while CNNs are also feedforward but share weights across spatial positions. RNNs add feedback connections for sequence processing, and transformer-based architectures, which underlie modern LLMs, are likewise feedforward and use attention to connect every token to every other (Vaswani et al., 2017, Sect. 3). Skip connections appear in these architectures as the residual connections of deep CNNs (He et al., 2016) and transformers (Vaswani et al., 2017). Each architecture fixes a different hypothesis space, within which empirical risk minimization learns a single hypothesis $\widehat{h} = h^{(\widehat{{\mathbf w}})}$ by choosing the model parameters $\widehat{{\mathbf w}}$ that minimize the average loss on a training set. The architecture also governs the generalization of the learned hypothesis to data points outside the training set, which is what the choice of an architecture is judged by. That choice need not be made by hand: neural architecture search learns a useful connectivity automatically (Elsken et al., 2019).

**See also:** [label](https://dictionaryofml.org/terms/label.html), artificial neuron, activation function, deep net, layer, convolutional neural network, recurrent neural network, [transformer](https://dictionaryofml.org/terms/transformer.html).

## References

- Goodfellow et al. (2016). Deep Learning. *MIT Press*.
- He et al. (2015). Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification. *2015 IEEE Int. Conf. Comput. Vis. (ICCV)*. <https://doi.org/10.1109/ICCV.2015.123>
- He et al. (2016). Deep Residual Learning for Image Recognition. *2016 IEEE Conf. Comput. Vis. Pattern Recognit.*. <https://doi.org/10.1109/CVPR.2016.90>
- Vaswani et al. (2017). Attention is all you need. *Adv. Neural Inf. Process. Syst.*. <https://papers.nips.cc/paper_files/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html>
- Elsken et al. (2019). Neural Architecture Search: A Survey. *J. Mach. Learn. Res.*.

## Cite this entry

```bibtex
@misc{dictml_ann,
  author = {Jung, Alexander},
  editor = {Olioumtsevits, Konstantina and Schnoor, Ekkehard},
  title = {artificial neural network (ANN)},
  howpublished = {Dictionary of Applied Machine Learning (course edition)},
  year = {2026},
  doi = {10.5281/zenodo.21569296},
  note = {ISBN 978-952-64-3013-3, CC BY 4.0, retrieved 2026-11-11},
  url = {https://dictionaryofml.org/terms/ann.html}
}
```