Learn Before
Disadvantages of Tuning-free Prompting
Because prompts are the only method that provides the task specification, heavy engineering is necessary to achieve high accuracy. In particular, in the in-context learning setting, providing many answered prompts can be slow at test time, and thus cannot easily use large training datasets.
0
1
Tags
Data Science
Related
Advantages of Promptless Fine-tuning
Disadvantages of Promptless Fine-tuning
Advantages of Tuning-free Prompting
Disadvantages of Tuning-free Prompting
Advantages of Fixed-LM Prompt Tuning
Disadvantages of Fixed-LM Prompt Tuning
Advantages of Fixed-prompt LM Tuning
Disadvantages of Fixed-prompt LM Tuning
Advantages of Prompt+LM Tuning
Disadvantages of Prompt+LM Tuning
Fine-tuning LLMs with Labeled Data
Standard Fine-Tuning
Selecting an Efficient Model Tuning Strategy
A key distinction between different methods for adapting a large language model is which components are modified versus which are kept fixed. Match each tuning strategy with the description of its core mechanism.
A research team is tasked with adapting a very large, pre-trained language model for a highly specialized task. They have access to a small, curated dataset of fewer than 100 examples. Their two main constraints are minimizing computational costs during the adaptation process and preventing the model from losing its extensive general-world knowledge. Which of the following adaptation strategies best balances these requirements?
Diagnosing and Correcting Model Tuning Issues