Input Vector Construction During Text Generation
Based on the scenario below, describe the two distinct pieces of information that are combined to form the initial input representation for the token 'quick' before it is processed by the model's internal layers. For each piece of information, explain the role it plays.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Application in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
An autoregressive language model is generating text one token at a time. It has just produced the token 'blue' as the fourth token in the sequence 'The sky is blue'. To determine the fifth token, the model must first create an input representation for the token 'blue' at position 4. How is this initial representation for 'blue' typically constructed before it is fed into the model's processing layers?
Input Vector Creation in Autoregressive Generation
Input Vector Construction During Text Generation