Learn Before
PGR Calculation Scenario
A research team is developing a new model. Their initial, 'weak' model achieves an accuracy of 50% on a specific classification task. They have a more powerful, 'strong' model architecture which, if trained on a perfect dataset, could theoretically reach a 'ceiling' accuracy of 90%. To save on labeling costs, they use the weak model to supervise the training of the strong model. After this process, the newly trained strong model achieves an accuracy of 80%. Based on this scenario, calculate the Performance Gap Recovered (PGR).
0
1
Tags
Ch.4 Alignment - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Application in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
PGR Calculation Scenario
In an experiment, a researcher observes that the performance of a strong model after being supervised by a weak one (P_weak→strong) is actually lower than the weak model's initial baseline performance (P_weak). Assuming the strong model's maximum potential performance (P_ceiling) is greater than the weak model's baseline, what is the resulting Performance Gap Recovered (PGR) and what does it signify?
Interpreting the PGR Formula's Denominator