An autoregressive language model is generating text one token at a time. It has just produced the token 'blue' as the fourth token in the sequence 'The sky is blue'. To determine the fifth token, the model must first create an input representation for the token 'blue' at position 4. How is this initial representation for 'blue' typically constructed before it is fed into the model's processing layers?
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Application in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
An autoregressive language model is generating text one token at a time. It has just produced the token 'blue' as the fourth token in the sequence 'The sky is blue'. To determine the fifth token, the model must first create an input representation for the token 'blue' at position 4. How is this initial representation for 'blue' typically constructed before it is fed into the model's processing layers?
Input Vector Creation in Autoregressive Generation
Input Vector Construction During Text Generation