In simple terms, an activation function is added in the neural network in order to help the network learn complex patterns in the data. When comparing this to the brain a neuron sends signals to other neurons but what signals being sent depends on the conditions. The activation function is at the end taking in the data from the node and transforms it into an input that a new node (neuron) can take into the new layer.

• And of course, typically math is involved with coming up with these functions.