Learn Before
Evaluating an LLM's Instruction-Following Performance
Analyze the provided model output in response to the user's prompt. Based on the model's ability to follow the given instructions, evaluate its performance and identify which specific constraints were not met.
0
1
Tags
Ch.4 Alignment - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Evaluation in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Evaluating an LLM's Instruction-Following Performance
A language model was given the following text and a specific set of instructions. Analyze the four responses below and identify which one demonstrates the best ability to follow all the given instructions.
Text: 'Solar energy is a renewable power source derived from the sun's radiation. It offers a significant advantage by reducing greenhouse gas emissions, which is crucial for combating climate change. Furthermore, solar panels can be installed on rooftops, making use of otherwise unused space and potentially lowering electricity bills for homeowners. The technology is continually improving, becoming more efficient and affordable.'
Instructions: 'Summarize the provided text in exactly three bullet points. Each bullet point must be a complete sentence and start with the phrase "The key takeaway is". Do not use the word "significant".'
Analyzing and Correcting Instruction-Following Failures