Learn Before
Evaluating an LLM Refinement Cycle
An LLM was tasked with explaining a scientific concept. Below are its initial response, the feedback it received, and its revised response. Evaluate how effectively the revision incorporates the given feedback. Identify one specific piece of feedback that was not fully addressed and suggest how the response could be further improved in the next iteration.
Initial Response: 'Photosynthesis is how plants eat sunlight to make their food.'
Feedback Provided:
- Avoid anthropomorphism; plants don't 'eat'. Use more precise scientific language.
- Include the key inputs (water, carbon dioxide) and outputs (glucose, oxygen).
- Mention where this process occurs within the plant cell.
Revised Response: 'Photosynthesis is a process where plants use sunlight, water, and carbon dioxide to create glucose, which is their food, and release oxygen as a byproduct.'
0
1
Tags
Ch.3 Prompting - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Evaluation in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Crafting a Revision Prompt
A user is refining an AI's response about the benefits of remote work. Below are the initial response, the feedback received, and the final refined response. Analyze the transformation and identify which piece of feedback was most responsible for the fundamental structural change between the two versions.
Initial Response: 'Remote work is good because you don't have to commute and you can have a flexible schedule.'
Feedback Provided:
- The response is too brief; it should be expanded with more detail.
- The benefits should be organized into categories, such as for employees and employers.
- A concluding sentence should be added to summarize the main points.
Refined Response: 'Remote work offers significant advantages for both employees and employers. For employees, key benefits include the elimination of daily commutes, leading to time and cost savings, and greater flexibility in managing work-life balance. For employers, the advantages include access to a wider talent pool and reduced overhead costs. In summary, the shift to remote work presents a compelling value proposition for both individuals and organizations.'
Evaluating an LLM Refinement Cycle