Learn Before
Triggering an External Calculator with Calculation Annotation
This technique integrates an external calculator during the prediction phase to improve the arithmetic accuracy of Large Language Models. The calculator is activated when the model generates output enclosed in specific annotation symbols, such as ≪ ... ≫. Within these annotated sections, the presence of an equals sign (=) signals the calculator to evaluate the expression preceding it, thereby offloading the computation and reducing the likelihood of errors.
0
1
Tags
Ch.3 Prompting - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Triggering an External Calculator with Calculation Annotation
Mechanism of Tool Invocation and Result Substitution
A user wants a language model to solve a word problem by showing its work and then providing a final answer. The model should format its calculations using
≪...≫so an external tool can execute them accurately, and use####to mark the final answer. Review the model's response below and identify the primary error in its formatting.Problem: A farmer has 3 fields, and each field produces 150 bushels of corn. If the farmer sells 200 bushels, how many are left?
Model's Response: First, I need to find the total production. ≪There are 3 fields, so 3 * 150 = 450 bushels total≫. Then, the farmer sells 200 bushels. The remaining amount is ≪450 - 200≫. So the final answer is 250.
Applying Calculation Annotation Formatting
A developer is designing a system where a Large Language Model solves math word problems. The system must ensure arithmetic accuracy by offloading calculations to an external tool and must also present the final answer in a clearly identifiable way. Which of the following model outputs best applies a formatting strategy to achieve both of these goals?
Learn After
A large language model is configured to offload arithmetic to an external tool. The tool is activated only when it detects an expression enclosed in
≪...≫that is immediately followed by an equals sign (=). Given this rule, analyze the following model outputs and identify which one would successfully trigger the external tool to perform the calculation.Debugging an LLM's Arithmetic Process
Constructing a Calculation-Enabled Model Output