Error signals are sent back through the network which influences how the connection weights are changed during the learning process. During training, the algorithm checks if the error is below the threshold. If so, the model can make a prediction if not the hyper parameters are updated and the model is training again.

Back Propagation (connectionism)

In machine learning, a distinction is made between a loss function and a cost function. A **loss function** (or error function) calculates the penalty for a single training example. For a given sample consisting of input features $x$ and a true label $y$, the loss function $L(\hat{y}, y)$ measures the discrepancy between the model's prediction $\hat{y}$ and the true label $y$. In contrast, a **cost function** (or objective function) is typically the average of the loss function values over the entire training set, providing an aggregate measure of the model's performance.

Loss Function vs. Cost Function

When the network generates wrong responses, an error signal is produced to help the learning process

University of Michigan - Ann Arbor

A network following the connectionist approach to cognitive modeling. Contains different kinds of units with connections between them. The units and connections are supposed to resemble neurons and their networks within the brain.

Connectionist network

Goldstein, E. B. (2014). Cognitive psychology: Connecting mind, research and everyday experience. Nelson Education.


Cognitive psychology: Connecting mind, research and everyday experience

Receive signals from the hidden units and generate some output based on what is received.

Output units (connectionism)

Receives signals from the input units and sends them to the output units.

Hidden units (connectionism)

Activated by stimuli and send signals to the hidden units.

Input units (connectionism)

Determines the extent to which a signal increases or decreases activity of the unit it was sent to.

connection weight

Artificial Neural Networks are built to resemble neurons in the human brain. The idea behind neural networks is to understand the pattern behind terms through examples rather than through task-specific rules.

Artificial Neural Networks

Example of activated connectionist network for living things and their properties

Error signal

When a part of a connectionist network is damaged, the entire system does not fail, and performance is only affected incrementally. This is similar to the brain, since someone with damage may retain all cognitive functionality aside from the functions that damaged area is specialized to.

Learn Before

Related

Learn After