A language model is generating a five-token sequence () using a permuted, non-sequential order. At a specific step in the generation process, the model calculates the probability for token as: , where is the embedding of token . Based only on this information, what can be definitively concluded about the generation process?
0
1
Tags
Ch.1 Pre-training - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A language model is tasked with generating a five-token sequence () in the specific permuted order: . At each step, the model predicts the next token in the permutation using the embeddings (e.g., for token ) of all previously generated tokens as context. Which of the following correctly represents the conditional probability for the third step of this generation process?
A language model generates a four-token sequence () using the specific permuted order: . Arrange the following conditional probability expressions to match this generation sequence, where represents the embedding of token .
A language model is generating a five-token sequence () using a permuted, non-sequential order. At a specific step in the generation process, the model calculates the probability for token as: , where is the embedding of token . Based only on this information, what can be definitively concluded about the generation process?