Learn Before
Commonsense Reasoning as a Challenging Task for LLMs
A significant challenge for Large Language Models is commonsense reasoning. This type of task requires the model to make logical inferences based on implicit, real-world knowledge that is not explicitly stated in the prompt, often leading to errors despite clear instructions.
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
GSM8K Benchmark
Insufficiency of Simple Demonstrations for LLM Reasoning Tasks
A user gives a language model the following prompt: 'I have a box that contains a red ball and a blue ball. I take the red ball out and put it on the table. What is left in the box?' The model responds: 'The box contains a red ball and a blue ball.' Which of the following best analyzes the likely cause of the model's incorrect answer?
Commonsense Reasoning as a Challenging Task for LLMs
In-Context Learning (ICL)
The Challenge of Multi-Step Logical Inference for LLMs in Arithmetic Reasoning
Language Model Scheduling Error Analysis
Predicting LLM Reasoning Flaws
Learn After
A researcher is designing a test to specifically evaluate a Large Language Model's commonsense reasoning capabilities, which rely on implicit, real-world knowledge not explicitly stated in the prompt. Which of the following prompts would be the most effective for this specific purpose?
Analysis of a Commonsense Reasoning Failure
Evaluating an LLM's Commonsense Reasoning Failure