Using Small Models for Pre-training or Fine-Tuning
One major approach to improving Large Language Models (LLMs) with smaller models focuses on the training phase, specifically using smaller models during the fine-tuning or pre-training of larger models. Key strategies in this category include using the small model to generate synthetic data for the large model to learn from, using the small model to select and filter appropriate high-quality data from a larger pool, and incorporating an auxiliary loss (such as knowledge distillation loss) based on the small model's outputs into the large model's training objective.
0
1
Tags
Foundations of Large Language Models
Ch.4 Alignment - Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Example of Successful Weak-to-Strong Generalization: GPT-4 with GPT-2 Supervision
Weak Performance (Pweak) as a Baseline Metric
Weak-to-Strong Performance (Pweak→strong)
Strong Ceiling Performance (Pceiling)
Performance Gap Recovered (PGR)
Data Selection and Filtering Using Weak Models
Cascading Inference
Weak-to-Strong Generalization via Fine-Tuning on Weak Model Data
AI System Optimization Strategy
An AI development team is building a system to answer a very high volume of customer support queries. They implement a two-step process: first, a small, fast model attempts to answer each query. If this model's confidence in its answer is low, the query is then passed to a much larger, more powerful, but slower model. What is the most significant strategic advantage of this architectural choice?
Direct Supervision via Knowledge Distillation Loss in Weak-to-Strong Generalization
When a large, powerful computational model is trained using labels generated exclusively by a smaller, less accurate model, the performance of the large model on new, unseen data is fundamentally limited and cannot exceed the accuracy of the smaller model that provided the training labels.
Using Small Models for Pre-training or Fine-Tuning
Combining Small and Large Models