Loss Function vs. Cost Function
In machine learning, a distinction is made between a loss function and a cost function. A loss function (or error function) calculates the penalty for a single training example. For a given sample consisting of input features and a true label , the loss function measures the discrepancy between the model's prediction and the true label . In contrast, a cost function (or objective function) is typically the average of the loss function values over the entire training set, providing an aggregate measure of the model's performance.
0
0
Contributors are:
Who are from:
Tags
Data Science
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Learn After
Sample-wise Negative Log-Likelihood Loss for a Sub-sequence
Sequence-Level Loss
An engineer is training a model on a large dataset. They are monitoring two metrics:
- Metric A: A value calculated for each individual data sample. This value fluctuates significantly from one sample to the next.
- Metric B: A single, aggregate value calculated after the model has processed the entire training dataset. This value shows a steady, downward trend over multiple passes through the dataset.
Based on the standard terminology for measuring a model's performance, what is the most accurate way to classify these two metrics?
Interpreting Training Metrics
Match each term to its most accurate description regarding how a model's performance is measured during training.