Learn Before
Delta Tuning
A more efficient method for fine-tuning parameters of very large-scale models. \By fine-tuning only a fraction of the parameters of a very large model, the cost of computation and storage is greatly reduced.
0
1
Tags
Deep Learning
Data Science
Related
Parameter-efficient Multi-task Fine-tuning for Transformers via Shared Hypernetworks
Delta Tuning
Instruction Fine-Tuning
Selecting an Efficient Fine-Tuning Strategy
A research lab needs to adapt a single, very large pre-trained language model (100B+ parameters) for 50 different, highly specialized downstream tasks. Their primary constraint is minimizing storage and computational costs, as creating and storing 50 full copies of the fine-tuned model is not feasible. Which fine-tuning strategy would be the most effective solution to this specific problem?
A development team is exploring different methods to adapt a large pre-trained language model for various applications. Match each of the following scenarios with the most appropriate fine-tuning strategy.