In-Context Learning (ICL)
In-context learning (ICL) is a method for improving the performance of Large Language Models by providing demonstrations within the prompt. A demonstration consists of an example problem and its corresponding solution. By conditioning its predictions on these examples, the model learns to follow the demonstrated problem-solving pattern for a given task without requiring updates to its parameters.
0
1
References
Reference of Foundations of Large Language Models Course
Reference of Foundations of Large Language Models Course
Reference of Foundations of Large Language Models Course
Reference of Foundations of Large Language Models Course
Reference of Foundations of Large Language Models Course
Reference of Foundations of Large Language Models Course
Tags
Ch.1 Pre-training - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Ch.3 Prompting - Foundations of Large Language Models
Ch.2 Generative Models - Foundations of Large Language Models
Related
Provided Answer (12) to the Example Arithmetic Reasoning Word Problem
Initial State for the Apple Problem
In-Context Learning (ICL)
A language model is presented with the following problem: "Jack has 7 apples. He ate 2 of them for dinner, but then his mom gave him 5 more apples. The next day, Jack gave 3 apples to his friend John. How many apples does Jack have left in the end?" The model processes the problem and performs the calculation
(7 + 5) - 2 - 3, arriving at the correct final answer of 7. Which of the following statements best analyzes the flaw in the model's problem-solving approach?A language model is tasked with solving the following word problem: 'Jack has 7 apples. He ate 2 of them for dinner, but then his mom gave him 5 more apples. The next day, Jack gave 3 apples to his friend John. How many apples does Jack have left in the end?' Arrange the following computational steps into the correct logical sequence that the model should follow to arrive at the final answer.
Analyzing a Flawed Arithmetic Reasoning Process
Incorrect Model Output () for the Jack's Apples Word Problem
Example of One-Shot Chain-of-Thought (COT) Prompting
Zero-Shot CoT Example with Jack's Apples
Learn After
Example of a Demonstration for In-Context Learning
Calculation Annotation in LLM Prompts
Example of a Demonstration for Sentiment Classification (Positive)
Example of a Demonstration for Sentiment Classification (Negative)
An AI developer provides a large language model with the following prompt: 'First, here are two examples of converting a sentence into a question. Example 1 Input: 'The cat is on the mat.' Example 1 Output: 'Is the cat on the mat?' Example 2 Input: 'They are running a race.' Example 2 Output: 'Are they running a race?' Now, using this pattern, convert the following sentence into a question: 'She is writing a book.' The model successfully outputs: 'Is she writing a book?' Which statement best analyzes the underlying mechanism that allowed the model to succeed?
Improving LLM Output Consistency
When a large language model successfully solves a new problem after being shown several examples within a single prompt, it is because the model's underlying weights have been permanently updated to incorporate the new problem-solving pattern.