Learn Before
Interpretation and Empirical Results of Performance Gap Recovered
A Performance Gap Recovered (PGR) of signifies that weak-to-strong fine-tuning completely bridges the performance gap between the weak baseline and the strong ceiling, while a PGR of means there is no improvement. Empirical studies, such as the research by Burns et al., have demonstrated that PGR can reach approximately {}0.8 across various NLP classification tasks. However, despite these promising indicators, realizing substantial weak-to-strong generalization continues to be a difficult objective requiring more research.
0
1
Tags
Foundations of Large Language Models
Ch.4 Alignment - Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Formula for Performance Gap Recovered (PGR)
An AI research team conducts two separate experiments to improve a powerful model's performance by having it learn from a less powerful one. The results are as follows:
- Experiment A: The less powerful model scores 50% on a task. The powerful model, after learning from the less powerful one, scores 70%. The powerful model's maximum possible score on this task is 90%.
- Experiment B: The less powerful model scores 70% on a different task. The powerful model, after learning from the
Evaluating Knowledge Transfer Effectiveness
Evaluating Performance Gains in Model Training
Interpretation and Empirical Results of Performance Gap Recovered