Learn Before
Visual Representation of Input Composition in Prompt Tuning
In prompt tuning, the input fed to the Large Language Model is a single sequence formed by concatenating trainable prompt embeddings with the standard token embeddings of the input text. This composition can be represented as the sequence {\mathbf{p}_0, \mathbf{p}_1, \dots, \mathbf{p}_n, \mathbf{e}_0, \mathbf{e}_1, \dots, \mathbf{e}_m}, where the {\mathbf{p}_i} vectors constitute the trainable soft prompt and the {\mathbf{e}_j} vectors are the embeddings for the input tokens. The prompt embeddings are learnable parameters adjusted for a specific task, while the token embeddings remain frozen.
0
1
Tags
Ch.3 Prompting - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Visual Representation of Input Composition in Prompt Tuning
A language model is being adapted for a new task. The input sequence for this model is constructed by prepending a series of 10 newly introduced, trainable vectors to the 50 standard, frozen word vectors that represent an input sentence. Which statement accurately analyzes the composition of the final 60-vector input sequence fed into the model?
A large language model is being adapted for a specific task by modifying its input. The final input sequence is created by combining a set of newly introduced, learnable vectors with the standard vectors representing the input text. Arrange these two components in the correct order to form the final sequence that is fed into the model.
Debugging an Input Composition Method
Learn After
An engineer is examining the input to a large language model that has been adapted for a new task. The input is visualized as a single sequence of vectors: [P1, P2, P3, P4, E1, E2, E3, E4, E5], where the 'P' vectors are adjusted during training and the 'E' vectors remain unchanged. Based on this structure, what is the most accurate analysis of this input method?
An engineer is designing the input for a large language model using a method where a small set of new, trainable vectors are prepended to the standard, frozen vectors of the input text. Below are four potential visual representations of this combined input sequence. Which diagram incorrectly illustrates this input composition?
Constructing a Prompt-Tuned Input Sequence