Critique of a Sequence Representation Method
A text-processing model operates sequentially from left to right, where the representation for any given word is calculated based only on the words that came before it. To create a single summary vector for an entire sentence, a colleague suggests using the model's output vector corresponding to the very first word. Analyze this suggestion and explain its primary flaw.
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Final Reward Score Calculation in RLHF
A team is building a system to evaluate text sequences. They use a model that processes text one token at a time from left to right, where the output for any given token is influenced only by the tokens that came before it. To obtain a single vector that represents an entire input sequence for scoring, which of the following strategies is most appropriate for this type of model?
Reward Model Implementation Analysis
Critique of a Sequence Representation Method