Learn Before
A language model developer is pre-training a model with the specific goal of improving its ability to identify and correct sentences containing incorrect word choices (e.g., distinguishing between 'your' and 'you're'). The model is trained to reconstruct the original, correct sentence from a deliberately damaged version. Which of the following input damage strategies would be most effective for this specific training objective?
0
1
Tags
Ch.1 Pre-training - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A language model developer is pre-training a model with the specific goal of improving its ability to identify and correct sentences containing incorrect word choices (e.g., distinguishing between 'your' and 'you're'). The model is trained to reconstruct the original, correct sentence from a deliberately damaged version. Which of the following input damage strategies would be most effective for this specific training objective?
Comparing Input Corruption Strategies
Evaluating Input Corruption Strategies for Typo Resilience