# sigmoid function

*Entry of the [Dictionary of Applied Machine Learning](https://dictionaryofml.org) (course edition), by Alexander Jung, Aalto University. CC BY 4.0. Updated 2026-11-08. [HTML](https://dictionaryofml.org/terms/sigmoid.html) · [typeset PDF](https://dictionaryofml.org/terms/sigmoid.pdf) · [LaTeX](https://dictionaryofml.org/terms/sigmoid.tex) · [Python demo](https://dictionaryofml.org/terms/sigmoid.py) · [notebook](https://dictionaryofml.org/terms/sigmoid.ipynb)*

Math is LaTeX with the book's macros expanded, so no notation package is needed to read it.

## Abstract

The sigmoid function maps the real line into the interval between $0$ and $1$. It translates the real-valued score of a binary classifier into an estimate for the probability of a specific label value. It is smooth and strictly increasing, with a derivative that vanishes for an argument with large magnitude. Machine learning uses it in two roles: as an activation function inside a neuron, and as part of the log-likelihood function for learning the model parameters of a probabilistic model.

## Definition

A frost warning for Krems an der Donau has to say how likely frost is, not only whether it is expected. Fig. 1 shows minimum daytime temperatures measured at that station. A classifier that warns of frost reads the recent measurements of a day, its feature vector ${\mathbf x}$, and delivers a real-valued score $h({\mathbf x})$. The classification is a threshold: a frost warning for the next day is issued if and only if $h({\mathbf x}) > 0$, and the magnitude $|h({\mathbf x})|$ measures the confidence in that warning. In practice it is often more useful to state a probability. The sigmoid function translates the score $h({\mathbf x})$ into one, as Fig. 1 shows for its last day.

![Figure 1 of the entry sigmoid](https://dictionaryofml.org/terms/fig/sigmoid-fig1.svg)

*Figure 1: Six consecutive days at Krems an der Donau, each stem the lowest temperature measured that day. The measurements a classifier reads for the last of them are its feature vector ${\mathbf x}$, from which the classifier computes the score $h({\mathbf x}) = 0.26$. That score is positive, so a frost warning is issued, and the sigmoid function turns it into $\sigma(0.26) = 0.57$, a $57$ percent probability of frost on 17/01. Data generated by pythondemos/sigmoid.py*

The sigmoid function maps the real line into the interval between $0$ and $1$. It is the function $\sigma: \mathbb{R} \rightarrow (0,1)$ given by

$$
\sigma(z) := \frac{1}{1 + e^{-z}}
\text{.}
$$

It rises from $\sigma(z) \rightarrow 0$ as $z \rightarrow -\infty$ to $\sigma(z) \rightarrow 1$ as $z \rightarrow +\infty$, and passes through $\sigma(0) = 1/2$ (see Fig. 2). It is also called the logistic function, and it is the softmax function for two classes.

![Figure 2 of the entry sigmoid](https://dictionaryofml.org/terms/fig/sigmoid-fig2.svg)

*Figure 2: The sigmoid function $\sigma(z) = 1/(1 + e^{-z})$ maps the real line into the open interval $(0,1)$, with $\sigma(0) = 1/2$*

The sigmoid function is smooth and strictly increasing, and it satisfies the symmetry $\sigma(-z) = 1 - \sigma(z)$ (Bishop, 2006, Sect. 4.2). Its derivative is available in closed form, and in terms of the function itself (Bishop, 2006, Sect. 4.3.2, Eq. (4.88)),

$$
\sigma'(z) = \sigma(z)\big(1 - \sigma(z)\big)
\text{.}
$$

These properties make the sigmoid function a popular choice for the activation function of an artificial neural network. One drawback of that choice is that the derivative decreases quickly as the argument grows in magnitude, which slows the progress of gradient-based learning methods, the vanishing gradient problem. That is why the rectified linear unit replaced the sigmoid in the hidden layers of deep nets (Goodfellow et al., 2016, Sect. 6.3.2).

A probabilistic model for a binary label uses the sigmoid function to turn a score into a probability. Let the label $y$ be either $1$ or $-1$, and let the score be ${\mathbf w}^{\top} {\mathbf x}$ with model parameters ${\mathbf w}$. The model reads

$$
\mathbb{P}\left(y = 1 \mid {\mathbf x}\right)
= \sigma\big({\mathbf w}^{\top} {\mathbf x}\big)
\text{,} \qquad
\mathbb{P}\left(y = -1 \mid {\mathbf x}\right)
= \sigma\big(-{\mathbf w}^{\top} {\mathbf x}\big)
\text{,}
$$

where the second follows from the symmetry $\sigma(-z) = 1 - \sigma(z)$. Both cases are the single expression $\sigma(y \, {\mathbf w}^{\top} {\mathbf x})$, which is where the sigmoid function enters the log-likelihood. For a training set $\mathcal{D}^{(\rm train)} = \big\{ \left( {\mathbf x}^{(r)},y^{(r)} \right) \big\}_{r=1}^{m}$ of labeled data points, the log-likelihood of the model parameters is

$$
\sum_{r=1}^{m} \ln
\sigma\big( y^{(r)} {\mathbf w}^{\top}
{\mathbf x}^{(r)} \big)
\text{.}
$$

Maximizing it over ${\mathbf w}$ is maximum likelihood estimation, and it is logistic regression. Each summand is the negative of a logistic loss, so maximizing the log-likelihood and minimizing the average logistic loss are the same computation (Bishop, 2006, Sect. 4.3.2). For example, consider the classifier map $h$ learned by logistic regression from the days of $2024$ at Krems. Where its score is zero the sigmoid returns $1/2$, so the warning is issued exactly where the probability passes one half. A score of $2.14$ becomes a probability of $0.89$ for frost tomorrow, and a score of $-2.67$ becomes $0.07$.

**Synonyms:** logistic function.

**See also:** activation function, rectified linear unit, softmax function, [logistic regression](https://dictionaryofml.org/terms/logreg.html), linear model, classifier, probability, smooth.

## References

- Bishop (2006). Pattern Recognition and Machine Learning. *Springer Science+Business Media*. <https://doi.org/10.1007/978-0-387-45528-0>
- Goodfellow et al. (2016). Deep Learning. *MIT Press*.

## Cite this entry

```bibtex
@misc{dictml_sigmoid,
  author = {Jung, Alexander},
  editor = {Olioumtsevits, Konstantina and Schnoor, Ekkehard},
  title = {sigmoid function},
  howpublished = {Dictionary of Applied Machine Learning (course edition)},
  year = {2026},
  doi = {10.5281/zenodo.21569296},
  note = {ISBN 978-952-64-3013-3, CC BY 4.0, retrieved 2026-11-11},
  url = {https://dictionaryofml.org/terms/sigmoid.html}
}
```