Learn Before
Rationale for Combined Span Prediction Loss
A model designed to extract answer spans from a text calculates its training loss by summing the negative log-likelihoods of two separate predictions for each token: one for the probability of being the start of the answer and one for the probability of being the end. Explain the primary reason why it is necessary to calculate and combine the loss from both of these predictions, rather than relying on just the start-of-span prediction alone.
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Span Prediction Loss Formula
A question-answering model is being trained to identify a specific answer span within a passage. The model's training objective is to minimize a loss calculated from two separate predictions for each token: the probability of it being the start of the answer and the probability of it being the end. The total loss is calculated by summing the negative log-likelihoods from both prediction networks. In which of the following scenarios would the model incur the highest training loss for a single training example?
Analyzing Span Prediction Model Loss
Rationale for Combined Span Prediction Loss