University of Michigan - Ann Arbor

On translated versions of the MMLU benchmark, GPT-4 demonstrates robust cross-lingual understanding by outperforming the English-language benchmark scores of preceding state-of-the-art models (such as Chinchilla and PaLM) across the vast majority of evaluated languages. Notably, GPT-4 surpasses prior English-language models even in low-resource languages with limited pre-training data availability, such as Latvian, Welsh, and Swahili.

GPT-4 Multilingual Performance on MMLU

Match each component of the MMLU evaluation to its role or description in assessing GPT-4:

Assess whether the reviewer's hypothesis is supported by the evaluation results and explain how GPT-4's performance in these specific languages compares to the English scores of prior models.

Learn Before

Related