Only one  function in the linear type. As the name says, the function will be linear. Ex: f(x) = x. Hence the output will not be confined into a range. Simply enough, it returns what it gets as input.

Linear Activation Function - Identity

- Sigmoid or Logistic Activation Function
- Tanh Activation Function
- ReLU (Rectified Linear Unit) Activation Function
- Softmax Activation Function

Non-Linear Activation Functions

https://www.nbshare.io/notebook/751082217/Activation-Functions-In-Python/

More Activation Functions

Luckily it can be categorized into 2 types:

- Linear Activation Function

- Non-linear Activation Function

University of Minnesota - Twin Cities

In simple terms, an activation function is added in the neural network in order to help the network learn complex patterns in the data. When comparing this to the brain a neuron sends signals to other neurons but what signals being sent depends on the conditions. The activation function is at the end taking in the data from the node and transforms it into an input that a new node (neuron) can take into the new layer.

- And of course, typically math is involved with coming up with these functions.

Activation Functions

- Vanishing Gradient Problem
- Zero Centered
- Computational Expense
- Differentiable

Learn Before

Related

Learn After