Learn Before
In the training of GPT-4, what was the smallest compute budget—expressed as a fraction of the full budget—used on smaller models to accurately predict aspects of final performance?
0
1
Tags
Prep Sessions
Transformer Architecture and Large Language Model Capabilities @ University of Michigan - Ann Arbor
Ch.2 Model Scaling and Capability Evaluation - Transformer Architecture and Large Language Model Capabilities @ University of Michigan - Ann Arbor
Predictable Scaling and Compute Laws - Transformer Architecture and Large Language Model Capabilities @ University of Michigan - Ann Arbor
Frontier Model Dynamics: Scaling Laws, Calibration, and Post-Training Alignment @ University of Michigan - Ann Arbor
Ch.1 Scaling Dynamics - Frontier Model Dynamics: Scaling Laws, Calibration, and Post-Training Alignment @ University of Michigan - Ann Arbor
Predictable Scaling Laws and Performance Extrapolation - Frontier Model Dynamics: Scaling Laws, Calibration, and Post-Training Alignment @ University of Michigan - Ann Arbor
Frontier Foundation Models, Capability Evaluation, and Just-In-Time Agent Harnesses @ University of Michigan - Ann Arbor
Ch.1 Foundation Model Capabilities and Benchmarking - Frontier Foundation Models, Capability Evaluation, and Just-In-Time Agent Harnesses @ University of Michigan - Ann Arbor
Predictable Scaling Laws in Foundation Models - Frontier Foundation Models, Capability Evaluation, and Just-In-Time Agent Harnesses @ University of Michigan - Ann Arbor
Related
What foundational technical elements are required to ensure that model behavior follows consistent mathematical trajectories across multiple orders of magnitude?
True or False: In predictable scaling for GPT-4, model behavior across multiple orders of magnitude was forecasted by fitting exponential scaling functions to smaller models.
Discuss how predictable scaling was applied in the development of GPT-4. In your response, describe the core technical prerequisites and identify the specific performance metrics that were forecasted.
Related
What foundational technical elements are required to ensure that model behavior follows consistent mathematical trajectories across multiple orders of magnitude?
True or False: In predictable scaling for GPT-4, model behavior across multiple orders of magnitude was forecasted by fitting exponential scaling functions to smaller models.
Discuss how predictable scaling was applied in the development of GPT-4. In your response, describe the core technical prerequisites and identify the specific performance metrics that were forecasted.