Backpropagation is a systematic computational procedure for applying the chain rule to calculate gradients automatically. It operates by traversing a computational graph in a backwards direction—from the output loss back to the input parameters—multiplying matrices of partial derivatives at each step to determine how parameters affect the final output.

Backpropagation

Error signals are sent back through the network which influences how the connection weights are changed during the learning process. During training, the algorithm checks if the error is below the threshold. If so, the model can make a prediction if not the hyper parameters are updated and the model is training again.

University of Michigan - Ann Arbor

When the network generates wrong responses, an error signal is produced to help the learning process

Error signal

Determines the extent to which a signal increases or decreases activity of the unit it was sent to.

connection weight

Goldstein, E. B. (2014). Cognitive psychology: Connecting mind, research and everyday experience. Nelson Education.


Cognitive psychology: Connecting mind, research and everyday experience

Back Propagation (connectionism)

In machine learning, a distinction is made between a loss function and a cost function. A **loss function** (or error function) calculates the penalty for a single training example. For a given sample consisting of input features $x$ and a true label $y$, the loss function $L(\hat{y}, y)$ measures the discrepancy between the model's prediction $\hat{y}$ and the true label $y$. In contrast, a **cost function** (or objective function) is typically the average of the loss function values over the entire training set, providing an aggregate measure of the model's performance.

Learn Before

Related

Learn After