Learn Before
A language model is being trained on the sentence: 'The quick brown fox jumps over the lazy dog.' The model's primary purpose is to generate new text by predicting the next word in a sequence based only on the words that came before it. When the model is calculating the representation for the word 'jumps' during this process, which part of the sentence is it allowed to consider?
0
1
Tags
Deep Learning (in Machine learning)
Foundations of Large Language Models Course
Data Science
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Chain Rule of Probability for Auto-regressive Language Models
Permuted Language Modeling (PLM)
A language model is being trained on the sentence: 'The quick brown fox jumps over the lazy dog.' The model's primary purpose is to generate new text by predicting the next word in a sequence based only on the words that came before it. When the model is calculating the representation for the word 'jumps' during this process, which part of the sentence is it allowed to consider?
Permuted Language Modeling
Model Architecture Suitability for Sentiment Analysis
Rationale for Auto-Regressive Model Design in Text Generation