Learn Before
Evaluating Reasoning Step Quality
A language model is given a problem and generates two different solution sequences, A and B. Both sequences arrive at the correct final answer. Based on the principle of representing a solution as a clear sequence of reasoning steps, which sequence demonstrates a higher-quality reasoning process? Justify your choice, considering factors like clarity, verifiability, and how well each step is decomposed.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Evaluation in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A language model is asked to solve the following problem: 'A library has 6 bookshelves, and each bookshelf holds 30 books. If 45 books are checked out, how many books remain on the shelves?' The model represents its solution as a sequence of reasoning steps,
y = (a_1, a_2, a_3). Analyze the sequence below and identify the step where the reasoning first becomes incorrect.a_1: Calculate the total number of books: 6 bookshelves * 30 books/bookshelf = 180 books.a_2: Calculate the remaining books by adding the checked-out books to the total: 180 + 45 = 225 books.a_3: The final answer is 225 books.A language model is tasked with solving the following problem: 'A bakery starts the day with 120 cupcakes. They sell 45 cupcakes in the morning and then bake another 60. How many cupcakes do they have now?' The model generates the individual reasoning steps for its solution,
y = (a_1, a_2, a_3). Arrange the following steps in the correct logical order to form a coherent solution.Evaluating Reasoning Step Quality