Question: In a concise analytical response, explain what you can learn—and what you cannot conclude—from manually inspecting about 20 mistakes made by a classifier on reviewed development examples.

Sample answer: Inspecting about 20 mistakes by hand can help reveal recurring failure patterns and point to the main kinds of errors the model is making. That makes it useful as a first pass for deciding where to investigate further. However, a review this small does not justify precise estimates or a complete explanation of the model’s behavior, so the result should be treated as a rough indication rather than a final diagnosis.

Key points:

• About 20 mistakes reviewed manually
• Development examples
• Identification of recurring error patterns
• Only a rough indication, not a full accounting

Rubric: A strong response identifies the approximate sample size, the development-set context, the value of spotting major error patterns, and the limitation that the conclusion is only approximate.