Learn Before
Calculation Annotation in LLM Prompts
Calculation annotation is a formatting technique that uses special symbols to structure a Large Language Model's output, particularly for tasks requiring arithmetic. For instance, symbols like ≪ ... ≫ can be used to demarcate calculation steps intended for execution by an external tool like a calculator. This allows the model to show its reasoning while offloading the actual computation to ensure accuracy. Other tokens, such as ####, can be used to clearly separate the final answer from the reasoning process.
0
1
Tags
Ch.3 Prompting - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Example of a Demonstration for In-Context Learning
Calculation Annotation in LLM Prompts
Example of a Demonstration for Sentiment Classification (Positive)
Example of a Demonstration for Sentiment Classification (Negative)
An AI developer provides a large language model with the following prompt: 'First, here are two examples of converting a sentence into a question. Example 1 Input: 'The cat is on the mat.' Example 1 Output: 'Is the cat on the mat?' Example 2 Input: 'They are running a race.' Example 2 Output: 'Are they running a race?' Now, using this pattern, convert the following sentence into a question: 'She is writing a book.' The model successfully outputs: 'Is she writing a book?' Which statement best analyzes the underlying mechanism that allowed the model to succeed?
Improving LLM Output Consistency
When a large language model successfully solves a new problem after being shown several examples within a single prompt, it is because the model's underlying weights have been permanently updated to incorporate the new problem-solving pattern.
Learn After
Triggering an External Calculator with Calculation Annotation
Mechanism of Tool Invocation and Result Substitution
A user wants a language model to solve a word problem by showing its work and then providing a final answer. The model should format its calculations using
≪...≫so an external tool can execute them accurately, and use####to mark the final answer. Review the model's response below and identify the primary error in its formatting.Problem: A farmer has 3 fields, and each field produces 150 bushels of corn. If the farmer sells 200 bushels, how many are left?
Model's Response: First, I need to find the total production. ≪There are 3 fields, so 3 * 150 = 450 bushels total≫. Then, the farmer sells 200 bushels. The remaining amount is ≪450 - 200≫. So the final answer is 250.
Applying Calculation Annotation Formatting
A developer is designing a system where a Large Language Model solves math word problems. The system must ensure arithmetic accuracy by offloading calculations to an external tool and must also present the final answer in a clearly identifiable way. Which of the following model outputs best applies a formatting strategy to achieve both of these goals?