Question: In a concise analytical response, explain how the purpose of the dev set determines whether mislabeled labels deserve correction.

Sample answer: The dev set exists to help a team quickly evaluate algorithms and decide whether Algorithm A or Algorithm B is better. Therefore, finding mislabeled examples does not by itself establish that correction is worth the time. The relevant question is whether the fraction that is mislabeled impedes those evaluations and comparisons. If it does, correcting the labels is worthwhile because it restores the dev set's ability to support rapid algorithm decisions.

Key points:

• The dev set supports quick algorithm evaluation.
• It should help determine whether Algorithm A or B is better.
• The relevant criterion is whether the mislabeled fraction impedes that judgment.
• Correction is worthwhile when evaluation or comparison is impeded.

Rubric: A strong response identifies the dev set's purpose, distinguishes the presence of mislabeling from its effect on judgment, and states the source's condition for spending time on correction.