Learn Before
Explain why this researcher's proposed protocol conflicts with the format mandated by the MMLU benchmark, and identify what the model is required to produce instead.
0
1
Tags
Prep Sessions
Transformer Architecture and Large Language Model Capabilities @ University of Michigan - Ann Arbor
Ch.2 Model Scaling and Capability Evaluation - Transformer Architecture and Large Language Model Capabilities @ University of Michigan - Ann Arbor
Multilingual Language Understanding on MMLU - Transformer Architecture and Large Language Model Capabilities @ University of Michigan - Ann Arbor
Ch.3 Model Alignment and Safety - Transformer Architecture and Large Language Model Capabilities @ University of Michigan - Ann Arbor
Model Calibration and Confidence Degradation - Transformer Architecture and Large Language Model Capabilities @ University of Michigan - Ann Arbor
Related
Example of an MMLU Question (Abstract Algebra)
Match each MMLU benchmark component to its role or definition described in the course content.
Explain why this researcher's proposed protocol conflicts with the format mandated by the MMLU benchmark, and identify what the model is required to produce instead.
According to the course content, which broad category of tasks does the MMLU benchmark organize into a question-answering format?
What full name does the acronym "MMLU" represent?
Multilingual Benchmark Translation for LLM Evaluation