Because the output of sigmoid is in range (0, 1), which can be regarded as a probability output that can be easily converted to class values. 

Why should Sigmoid be used for the output layer?

- tanh works almost always better than Sigmoid function, but Sigmoid should be used for the output layer of binary classification models.
- tanh is in range (-1, 1), but Sigmoid is in range (0, 1).
- Using tanh activation functions in the hidden layers optimizes training because it centers the data around 0 rather tahn 0.5

University of Michigan - Ann Arbor

$S(x) = \frac{1}{1 +e^{-x}} = \frac{e^x}{e^x + 1}$ 


Sigmoid/Logistic Function

$S(x)  = tanh(x) = \frac{e^x-e^{-x}}{e^x + e^{-x}}$


TanH/Hyperbolic Tangent Function

Pros:
 - Smooth gradient, preventing “jumps” in output values.
 - Output values bound between 0 and 1, normalizing the output of each neuron.
 - Clear predictions—For X above 2 or below -2, tends to bring the Y value (the prediction) to the edge of the curve, very close to 1 or 0. This enables clear predictions.

Cons:
 - Vanishing gradient—for very high or very low values of X, there is almost no change to the prediction, causing a vanishing gradient problem. This can result in the network refusing to learn further, or being too slow to reach an accurate prediction.
 - Outputs not zero centered.
 - Computationally expensive


Pros and Cons of Sigmoid/Logistic Function

$$g(x) = \frac{1}{1 + e^{-x}} \Rightarrow g'(x) = \frac{d}{dx}g(x) = \frac{1}{1 + e^{-x}} (1 - \frac{1}{1 + e^{-x}}) = g(x) (1 - g(x))$$ 

Derivative of Sigmoid/Logistic Function

Sigmoid/Logistic vs. TanH/Hyperbolic Tangent functions

The numerical output. ofa sigmoid node in a neural netork

Question about sigmoid node

Pros:
 - Zero centered—making it easier to model inputs that have strongly negative, neutral, and strongly positive values.
 - Smooth gradient, preventing “jumps” in output values.
 - Output values bound between -1 and 1, normalizing the output of each neuron.
 - Clear predictions—For X above 2 or below -2, tends to bring the Y value (the prediction) to the edge of the curve, very close to 1 or -1. This enables clear predictions.

Cons (Like the Sigmoid function):
 - Vanishing gradient—for very high or very low values of X, there is almost no change to the prediction, causing a vanishing gradient problem. This can result in the network refusing to learn further, or being too slow to reach an accurate prediction.
 - Computationally expensive

Pros and Cons of Hyperbolic Tangent Function

$$g(x) = \frac{e^{x} - e^{-x}}{e^{x} + e^{-x}} \Rightarrow g'(x) = \frac{d}{dx}g(x) = 1 - (g(x))^{2}$$ 

Derivative of TanH/Hyperbolic Tangent Function

https://medium.com/the-theory-of-everything/understanding-activation-functions-in-neural-networks-9491262884e0

More on the Tanh function

You have built a network using the tanh activation for all the hidden units. You initialize the weights to relative large values, using np.random.randn(..,..)*1000. What will happen?

Learn Before

Related

Learn After