Learn Before
Debugging an Input Composition Method
A researcher is adapting a large language model for a new task. They are constructing the input sequence by first taking the user's text and converting it into its standard, fixed token embeddings. They then append a set of newly introduced, trainable vectors to the end of this sequence. Despite training, the model's performance is poor. Based on the standard method for composing inputs in this context, identify the fundamental error in the researcher's approach and explain the correct procedure.
0
1
Tags
Ch.3 Prompting - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Application in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Visual Representation of Input Composition in Prompt Tuning
A language model is being adapted for a new task. The input sequence for this model is constructed by prepending a series of 10 newly introduced, trainable vectors to the 50 standard, frozen word vectors that represent an input sentence. Which statement accurately analyzes the composition of the final 60-vector input sequence fed into the model?
A large language model is being adapted for a specific task by modifying its input. The final input sequence is created by combining a set of newly introduced, learnable vectors with the standard vectors representing the input text. Arrange these two components in the correct order to form the final sequence that is fed into the model.
Debugging an Input Composition Method