During neural network training, forward and backward propagation are fundamentally interdependent. Forward propagation depends on the current model parameters, which are updated by the optimization algorithm using gradients calculated during the most recent backpropagation step. Conversely, backpropagation traverses the computational graph in reverse and depends on the intermediate variables that were computed and stored during the forward propagation pass. For example, computing the regularization term during forward propagation depends on the current values of model parameters $$\mathbf{W}^{(1)}$$ and $$\mathbf{W}^{(2)}$$, which are given by the optimization algorithm according to backpropagation in the most recent iteration. On the other hand, the gradient calculation during backpropagation depends on the current value of the hidden layer output $$\mathbf{h}$$, which is given by forward propagation.

Interdependence of Forward and Backward Propagation

When working with Neural networks, it involves quite a bit a math to determine the weights and bias. This is so the computer can accurately determine the output. But in order to really figure that out. Fine tuning of the weights and bias from the input data is known as "training" the neural network.

University of Minnesota - Twin Cities

Breaking this down into to components:  

- What are Neurons? 
- How are they connected?


 - At first, we can think view a neural network as how our brains work like neurons signaling one another (Also the inspiration for its name). In a sense these models allow computer programs to recognize patterns and solve common problems.

- Neural Networks are comprised of node layers. It contains the input layer, one (or more) hidden layers, and an output layer.

- Nodes (neurons) are connected via arrows and each has an associated weight and threshold. If the node surpasses the threshold set, then that neuron is activated and the data is sent on the next layer. This process repeats until it goes to the output layer.


Neural Network


While there are many types of neural networks the most common networks that will be typically seen are: 

- Feedforward neural networks
- Convolutional neural networks
- Recurrent neural networks

Learn Before

Related

Learn After