GSM8K Benchmark

Insufficiency of Simple Demonstrations for LLM Reasoning Tasks

A user gives a language model the following prompt: 'I have a box that contains a red ball and a blue ball. I take the red ball out and put it on the table. What is left in the box?' The model responds: 'The box contains a red ball and a blue ball.' Which of the following best analyzes the likely cause of the model's incorrect answer?

Commonsense Reasoning as a Challenging Task for LLMs

In-Context Learning (ICL)

The Challenge of Multi-Step Logical Inference for LLMs in Arithmetic Reasoning

Language Model Scheduling Error Analysis

Predicting LLM Reasoning Flaws