Learn Before
Disadvantages of Fixed-LM Prompt Tuning
Not applicable in zero-shot scenarios. While effective in few-shot scenarios, representation power is limited in large-data settings. Prompt engineering through the choice of hyperparameters or seed prompts is necessary. Prompts are usually not human-interpretable or manipulable.
0
1
Tags
Data Science
Related
Advantages of Promptless Fine-tuning
Disadvantages of Promptless Fine-tuning
Advantages of Tuning-free Prompting
Disadvantages of Tuning-free Prompting
Advantages of Fixed-LM Prompt Tuning
Disadvantages of Fixed-LM Prompt Tuning
Advantages of Fixed-prompt LM Tuning
Disadvantages of Fixed-prompt LM Tuning
Advantages of Prompt+LM Tuning
Disadvantages of Prompt+LM Tuning
Fine-tuning LLMs with Labeled Data
Standard Fine-Tuning
Selecting an Efficient Model Tuning Strategy
A key distinction between different methods for adapting a large language model is which components are modified versus which are kept fixed. Match each tuning strategy with the description of its core mechanism.
A research team is tasked with adapting a very large, pre-trained language model for a highly specialized task. They have access to a small, curated dataset of fewer than 100 examples. Their two main constraints are minimizing computational costs during the adaptation process and preventing the model from losing its extensive general-world knowledge. Which of the following adaptation strategies best balances these requirements?
Diagnosing and Correcting Model Tuning Issues