Pros and Cons of Hyperbolic Tangent Function

Derivative of TanH/Hyperbolic Tangent Function

More on the Tanh function

You have built a network using the tanh activation for all the hidden units. You initialize the weights to relative large values, using np.random.randn(..,..)*1000. What will happen?

Sigmoid/Logistic vs. TanH/Hyperbolic Tangent Functions

Linear vs. Non-Linear Activation Functions

Sigmoid/Logistic Function

Swish Function

ReLU (Rectified Linear Unit)

ELU (Exponential Linear Unit)

Which activation function is represented by each of these plots?

Which of the following introduces nonlinearity into neural networks?

Softmax Function

Pros and Cons of Sigmoid, Tanh, and ReLU Activation Functions