Learn Before
Critique of an Early Transformer-Based Language Model
A pioneering 2018 language model was based on a transformer architecture that processed text strictly in a left-to-right sequence to predict the next word. Evaluate the primary conceptual limitation of this unidirectional approach. In your analysis, discuss specific types of language understanding tasks where this design choice would likely result in suboptimal performance and explain why.
0
1
Tags
Data Science
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Evaluation in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A foundational generative language model introduced in 2018 significantly improved the ability to capture relationships between words far apart in a text, a major challenge for previous sequential models. Which of the following best analyzes the core architectural innovation responsible for this leap in performance?
Critique of an Early Transformer-Based Language Model
Training Objective of an Early Transformer Model