University of Michigan - Ann Arbor

Neural networks are made up of nodes (each representing a Logistic regression) with layers. They are trained to map an input vector X to an output vector Y based on the training data. The hidden layers of a network contain neurons, connections, and activation functions that occur between the input and output. An activation function is applied at each layer to correctly connect the input and output. Different types of networks may use diverse types of activation functions to correctly identify the nature of how the input and output are related.
In the figure out of the 4 neurons, the Logistic regression representation of two of them are shown.

Artificial Neural Networks Formulation

Machine Learning Yearning, a free ebook from Andrew Ng, teaches you how to structure Machine Learning projects.
https://www.deeplearning.ai/machine-learning-yearning/

Note: The content of the book is aligned with the Coursera Deeplearning.ai specialization.  https://www.deeplearning.ai/deep-learning-specialization/ 

Machine Learning Yearning (Deeplearning.ai)

In a neural network, numeric data points, called inputs, are fed into the neurons in the input layer. Each neuron has a weight, and multiplying the input number with the weight gives the output of the neuron, which is transferred to the next layer.
The activation function is a mathematical “gate” in between the input feeding the current neuron and its output going to the next layer. It can be as simple as a step function that turns the neuron output on and off, depending on a rule or threshold. Or it can be a transformation that maps the input signals into output signals that are needed for the neural network to function.
Activation functions are mathematical equations that determine the output of a neural network. The function is attached to each neuron in the network, and determines whether it should be activated (“fired”) or not, based on whether each neuron’s input is relevant for the model’s prediction. Activation functions also help normalize the output of each neuron to a range between 1 and 0 or between -1 and 1. When used in a multilayer neural network, activation functions can be different for different layers. 

Learn Before

Related