Conceptual Parallel in Generative Modeling
Scenario A: A text-infilling model is tasked with completing the sentence 'The dog ___ in the park.' It replaces the blank, which could represent any number of words (e.g., 'played', 'was chasing a ball'), with a single special symbol. The model must then generate an appropriate sequence of words to replace that one symbol.
Scenario B: An early machine translation system is translating a French sentence containing the word 'au' into English. Depending on the context, this single French word might need to be translated into two English words, 'to the'.
Explain the fundamental modeling challenge that is common to both of these scenarios.
0
1
Tags
Ch.1 Pre-training - Foundations of Large Language Models
Foundations of Large Language Models
Computing Sciences
Foundations of Large Language Models Course
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Conceptual Parallel in Generative Modeling
In generative language models, the challenge of predicting the length of a masked span is often compared to the concept of 'fertility' from statistical machine translation. Which of the following statements most accurately analyzes the relationship between these two concepts?
Critique of the Fertility Modeling Analogy