Learn Before
Example of an MMLU Question (Abstract Algebra)
An example of a question from the MMLU benchmark, taken from the abstract algebra subject, is presented as follows:
is a group with for all , in . The inverse of is (A) (B) (C) (D) D
This illustrates the multiple-choice format used in the benchmark to test a model's reasoning capabilities on specialized topics.
0
1
Tags
Ch.3 Prompting - Foundations of Large Language Models
Foundations of Large Language Models
Computing Sciences
Foundations of Large Language Models Course
Learn After
Analysis of a Reasoning-Based Benchmark Question
Consider the following problem, which is an example from a benchmark designed to test the reasoning capabilities of advanced AI models:
A mathematical structure is defined by the set of all integers and an operation
*such that for any two integersaandb,a * b = a + b + 1. This structure forms a group. What is the inverse of an elementawithin this group?Analysis of a Benchmark Reasoning Problem