Improving a Preference Labeling Prompt with Chain-of-Thought
You are creating a set of instructions for data labelers. Below is a user query, two AI-generated responses, and a simple preference explanation. Your task is to rewrite the explanation to include a clear, step-by-step 'chain-of-thought' rationale that a labeler could use as a model for their own work.
User Query: "Explain the water cycle for a 5th grader."
Response A: "The water cycle is the process of water moving around the Earth. It includes evaporation, condensation, and precipitation. Water evaporates from oceans, forms clouds through condensation, and then falls back as rain, which is precipitation."
Response B: "Imagine a big puddle on a sunny day! The sun heats the water, and it turns into an invisible gas called water vapor, floating up into the sky. This is called evaporation. High up, where it's colder, the water vapor turns back into tiny liquid water droplets, forming clouds. This is condensation. When the clouds get too full of water, the water falls back down as rain, snow, or hail. This is precipitation. The water collects in rivers and oceans, and the whole adventure starts again!"
Initial Explanation: "Response B is better because it's more engaging."
Your Task: Rewrite the initial explanation to demonstrate a chain-of-thought process.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Computing Sciences
Foundations of Large Language Models Course
Application in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Diagnosing Inconsistent Preference Labeling
A research team is creating instructions for human labelers who will be rating the quality of two different AI-generated responses to a user's query. The team wants to include an example in their instructions that not only shows a preference but also models a clear, step-by-step reasoning process to guide the labelers. Which of the following examples best accomplishes this goal?
Improving a Preference Labeling Prompt with Chain-of-Thought