Optimizing Annotation Strategy for a Reasoning Model
To achieve the greatest improvement in the model's reasoning ability, which group of incorrect steps (A or B) should the team prioritize for their annotation efforts? Explain your reasoning.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Application in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Optimizing Annotation Strategy for a Reasoning Model
A team is refining a language model that generates step-by-step solutions to complex problems. For each reasoning step, the model provides a confidence score indicating its certainty in the step's correctness. The team has a limited budget for human annotators to review and correct the model's reasoning. To maximize the model's performance improvement with this limited budget, which of the following types of reasoning steps should the team prioritize for annotation?
Evaluating Annotation Strategies for Model Refinement