Question: In a 1,000-example dev set with 98.0% accuracy, explain why practical convenience—rather than a deliberate methodological decision—leads teams to introduce label-correction bias.

Sample answer: With 98.0% accuracy on 1,000 examples, only 20 examples are misclassified while 980 are correct. Reviewing 20 examples takes far less time and effort than reviewing 980, so teams naturally gravitate toward checking the smaller, misclassified set. This is a practical shortcut driven by workload, not a considered decision about which examples are most likely to contain labeling errors. As a result, any label corrections applied only to the misclassified group can skew the dev set's label distribution in ways the team did not intend.

Key points:

• 1,000-example dev set at 98.0% accuracy yields 20 misclassified and 980 correctly classified examples
• Reviewing 20 examples requires far less effort than reviewing 980
• The bias arises from practical convenience, not intentional methodology
• Unreviewed correct examples may still contain label errors that go uncorrected

Rubric: Full credit requires identifying the 20 vs. 980 example split, explaining that the smaller misclassified group is easier to review, and noting that this convenience-driven behavior is unintentional rather than a deliberate methodological choice.