From the figure it can be seen that the sigmoid function becomes flat when its argument is highly positive or highly negative.

Graph of Sigmoid Function

Sigmoid function defines the $\phi$ parameter of Bernoulli distribution. Sigmoid function can depict any real value in the range between 0 and 1.

$\sigma(x) = \frac{1}{1+e^{-x}}$


San Jose State University

There are some functions used while describing probability distribution in deep learning models they are
- Logistic Sigmoid
- Softplus Function


Properties of Common Functions

Goodfellow, I., Bengio, Y., & Courville, A. (2016). $\mathit{Deep \ Learning.}$ MIT Press. Retrieved from [www.deeplearningbook.org](https://www.deeplearningbook.org) 

Deep Learning

Logistic Sigmoid

Softplus function defines $\beta$ or $\sigma$ of normal distribution since they have a range from 0 to $\infty.

$\zeta(x) = log(1+e^x)$


Learn Before

Related

Learn After