Case context: A team uses a dev set to compare Algorithm A with Algorithm B. The team finds that mislabeled dev examples prevent it from confidently deciding which algorithm is better and slow its evaluation process.

Question: Based only on the source criterion, what should the team decide, and why?

Sample answer: The team should spend time correcting the mislabeled dev set labels. The errors are impeding both quick evaluation and the judgment of whether Algorithm A or Algorithm B is better, which is precisely the condition under which the source says correction is worthwhile.

Key points:

• The dev set is used to compare algorithms.
• Mislabeled examples are impeding the comparison.
• Evaluation is no longer quick or decisive.
• The source therefore supports correcting the labels.

Rubric: The response should recommend fixing the labels and connect that decision directly to the impaired speed or reliability of the A-versus-B evaluation.