Learn Before
In the training of GPT-4, what was the smallest compute budget—expressed as a fraction of the full budget—used on smaller models to accurately predict aspects of final performance?
0
1
Tags
Prep Sessions
Transformer Architecture and Large Language Model Capabilities @ University of Michigan - Ann Arbor
Ch.2 Model Scaling and Capability Evaluation - Transformer Architecture and Large Language Model Capabilities @ University of Michigan - Ann Arbor
Predictable Scaling and Compute Laws - Transformer Architecture and Large Language Model Capabilities @ University of Michigan - Ann Arbor
Frontier Model Dynamics: Scaling Laws, Calibration, and Post-Training Alignment @ University of Michigan - Ann Arbor
Ch.1 Scaling Dynamics - Frontier Model Dynamics: Scaling Laws, Calibration, and Post-Training Alignment @ University of Michigan - Ann Arbor
Predictable Scaling Laws and Performance Extrapolation - Frontier Model Dynamics: Scaling Laws, Calibration, and Post-Training Alignment @ University of Michigan - Ann Arbor
Frontier Foundation Models, Capability Evaluation, and Just-In-Time Agent Harnesses @ University of Michigan - Ann Arbor
Ch.1 Foundation Model Capabilities and Benchmarking - Frontier Foundation Models, Capability Evaluation, and Just-In-Time Agent Harnesses @ University of Michigan - Ann Arbor
Predictable Scaling Laws in Foundation Models - Frontier Foundation Models, Capability Evaluation, and Just-In-Time Agent Harnesses @ University of Michigan - Ann Arbor
Related
What foundational technical elements are required to ensure that model behavior follows consistent mathematical trajectories across multiple orders of magnitude?
In the training of GPT-4, what was the smallest compute budget—expressed as a fraction of the full budget—used on smaller models to accurately predict aspects of final performance?
True or False: In predictable scaling for GPT-4, model behavior across multiple orders of magnitude was forecasted by fitting exponential scaling functions to smaller models.
Discuss how predictable scaling was applied in the development of GPT-4. In your response, describe the core technical prerequisites and identify the specific performance metrics that were forecasted.