Analyzing a Permuted Prediction Task
A language model is processing a sentence in a permuted order, indicated by the numbers in parentheses: jumps(3) over(8) The(1) lazy(5) dog(6) . (7) quick(2) brown(4). The model's task is to predict the word at position 4. It is given the words at positions 1, 2, and 3 as context. Based on this information, what word is the model trying to predict, and why is this specific context sufficient for this task?
0
1
Tags
Ch.1 Pre-training - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A language model receives the following input, where each word is paired with a number indicating its position in a shuffled sequence:
quick4 The1 brown2 fox5 jumps3. Based on these numerical positions, what is the actual sequence of words the model processes?Analyzing a Permuted Prediction Task
A language model is given the following tokens from a sentence, where each word is paired with a number. This number indicates the order in which the model should process the token. Arrange the tokens below into the correct processing sequence, from the first to be processed to the last.