Learn Before
Evaluating a Novel Prompting Strategy
Based on your understanding of how to elicit better reasoning from language models, evaluate the likely effectiveness of the 'Reflective Rationale Prompting' strategy for improving the accuracy of the initial answer. Justify your evaluation.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Evaluation in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Evaluating a Novel Prompting Strategy
A researcher is trying to get a language model to solve a multi-step logic puzzle. They test two different prompts:
Prompt A: 'What is the solution to the following logic puzzle? [Puzzle text]'
Prompt B: 'Solve the following logic puzzle. First, break down the puzzle into individual facts and constraints. Next, reason through the implications of each fact step-by-step. Finally, state your conclusion and explain how you arrived at it. [Puzzle text]'
Which statement best analyzes why Prompt B is likely to yield a more accurate solution for this type of task?
Evaluating LLM Reasoning Outputs
Explicit Prompting for Extended Deliberation
Modifying Decoding for Longer Reasoning Paths
Multi-Stage Generation for Incremental Reasoning