Visual Diagram of Soft Prompt Training

A developer is adapting a large, pre-trained language model for a new task by adding a small set of learnable, continuous vector parameters to the input. During the training process, for each example, a loss is computed by comparing the model's output to the correct output. According to the standard supervised learning approach for this technique, how is this loss used to update the system's parameters?

A machine learning engineer is using a supervised learning approach to train a set of continuous, learnable prompt parameters for a large, pre-trained language model. The goal is to adapt the model for a specific task. During each training step, a loss is calculated based on the difference between the model's prediction and the correct output. Which of the following statements most accurately describes how the system's parameters are handled during this process?

Diagnosing a Soft Prompt Training Issue