State Evolution in Token Generation
A language model is given the initial input x = 'The best way to learn is by'. It then generates two tokens, one after the other: 'doing' and then '.'. Describe the model's state after it generates the token 'doing', and then describe how the state changes when it subsequently generates the token '.' based on the formal state representation as a pair of the initial input and the generated sequence.
0
1
Tags
Ch.4 Alignment - Foundations of Large Language Models
Foundations of Large Language Models
Computing Sciences
Foundations of Large Language Models Course
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A language model is provided with the initial input sequence 'x' = "The sun is shining and the". The model then generates a sequence of three tokens, 'yt' = "sky is blue". According to the formal state definition where the state is a pair of the initial input and the generated sequence, what is the correct representation of the model's state at this point?
Analyzing Language Model State Evolution
State Evolution in Token Generation