Question: When auditing a validation set for labeling quality, explain why looking only at examples your model got wrong is not enough. Why should some examples the model got right also be reviewed?

Sample answer: Reviewing only the model’s errors can hide cases where the stored label is wrong and the model happened to make the same wrong judgment. In that situation, the prediction and the label agree, so the example appears correct even though the ground truth is faulty. Checking some correctly classified examples helps reveal these silent label mistakes and verifies that agreement with the model really means the label is trustworthy.

Key points:

• Error-only review can miss hidden label defects.
• A prediction can match an incorrect label by coincidence.
• Correctly classified examples should also be sampled.
• The goal is to validate the labels, not just the model’s disagreements.

Rubric: The answer should explain that some examples are mislabeled even when the model predicts the same class, so reviewing only misclassifications misses those cases. It should also state that checking correctly classified items helps confirm label accuracy.