Learn Before
Academic and Professional Exam Benchmarks - Transformer Architecture and Large Language Model Capabilities @ University of Michigan - Ann Arbor
Impact of RLHF on Model Capability - Transformer Architecture and Large Language Model Capabilities @ University of Michigan - Ann Arbor
GPT-4 Performance on Academic and Professional Exams
RLHF Effects on Capability and Calibration - Frontier Foundation Models, Capability Evaluation, and Just-In-Time Agent Harnesses @ University of Michigan - Ann Arbor
Origin of GPT-4 Exam Capabilities in Pre-training
Empirical evaluation demonstrates that GPT-4's performance across standardized exam benchmarks originates primarily in the pre-training stage rather than post-training alignment. When tested on multiple-choice exam sections, the base pre-trained GPT-4 model achieves an average score of $73.7%while the post-RLHF model scores \74.0%$, indicating that reinforcement learning from human feedback does not substantially alter the fundamental capabilities acquired during pre-training.
0
1
Tags
Prep Sessions
Transformer Architecture and Large Language Model Capabilities @ University of Michigan - Ann Arbor
Ch.2 Model Scaling and Capability Evaluation - Transformer Architecture and Large Language Model Capabilities @ University of Michigan - Ann Arbor
Academic and Professional Exam Benchmarks - Transformer Architecture and Large Language Model Capabilities @ University of Michigan - Ann Arbor
Ch.3 Model Alignment and Safety - Transformer Architecture and Large Language Model Capabilities @ University of Michigan - Ann Arbor
Impact of RLHF on Model Capability - Transformer Architecture and Large Language Model Capabilities @ University of Michigan - Ann Arbor
Frontier Foundation Models, Capability Evaluation, and Just-In-Time Agent Harnesses @ University of Michigan - Ann Arbor
Ch.2 Post-Training Alignment and Calibration - Frontier Foundation Models, Capability Evaluation, and Just-In-Time Agent Harnesses @ University of Michigan - Ann Arbor
RLHF Effects on Capability and Calibration - Frontier Foundation Models, Capability Evaluation, and Just-In-Time Agent Harnesses @ University of Michigan - Ann Arbor
Related
GPT-4 Performance on Academic and Professional Exams
Origin of GPT-4 Exam Capabilities in Pre-training
GPT-4
Origin of GPT-4 Exam Capabilities in Pre-training
Evaluation Asymmetry in Base and RLHF Free-Response Comparison
How did GPT-4's performance compare to GPT-3.5 on the simulated Uniform Bar Examination?
The academic and professional examinations used to evaluate GPT-4 share which design characteristic?
Aside from the Uniform Bar Examination, which four standardized examination programs are explicitly cited as part of GPT-4's evaluation?
Origin of GPT-4 Exam Capabilities in Pre-training
What level of performance did GPT-4 achieve on the majority of the academic and professional exams on which it was evaluated?
Degradation of Model Calibration from Post-Training Alignment
Origin of GPT-4 Exam Capabilities in Pre-training
Evaluation Asymmetry in Base and RLHF Free-Response Comparison
Learn After
Empirical evaluation shows that GPT-4's standardized exam benchmark capabilities originate primarily from post-training alignment rather than pre-training.
In empirical evaluations of GPT-4, which finding provides evidence that standardized exam performance originates primarily in pre-training rather than post-training alignment?
On what specific format of standardized exam sections were the base pre-trained and post-RLHF GPT-4 models evaluated to compare their benchmark capabilities?
Evaluation Asymmetry in Base and RLHF Free-Response Comparison
Match each standardized exam benchmark metric with its corresponding empirical result for GPT-4.
Which section format was used in the empirical evaluations comparing the standardized benchmark performance of base and post-RLHF GPT-4?
In which stage of model development does GPT-4's performance across standardized exam benchmarks primarily originate?