Learn Before
Choosing a Fine-Tuning Strategy
A startup is considering a tuning method where a large, pre-trained model's original parameters are kept frozen. For each new customer task, a small set of new, trainable vectors is created and prepended to the hidden states at each layer of the model to guide its behavior. Based on the startup's constraints described in the case study, evaluate the suitability of this proposed method and justify your conclusion.
0
1
Tags
Ch.3 Prompting - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Evaluation in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Architecture of Prefix Tuning
A team is tasked with adapting a very large, pre-trained language model for a specialized legal document analysis task. To conserve computational resources and avoid altering the base model, they freeze all of the original model's parameters. They then introduce a small set of new, trainable parameters that are prepended to the sequence of hidden states within each transformer layer. During training for the new task, only these new parameters are updated. Which statement best analyzes the main consequence of this specific training strategy?
Choosing a Fine-Tuning Strategy
Analyzing the Mechanism of Prefix Tuning