Question: In a family photo-sharing app that must block explicit uploads, explain why overall classification accuracy may fail to reflect the product’s true goal, and what this teaches about selecting evaluation metrics.

Sample answer: Accuracy counts every mistake in the same way, so it can hide the difference between a harmless error and a serious one. In this app, one model may score slightly better overall while still allowing a small number of explicit images to pass through. Even if those slips are rare, they can damage user trust and defeat the main purpose of the product. The lesson is that the evaluation metric should be tied to the business objective and the real cost of each kind of error, not just to the total number of correct predictions.

Key points:

• Accuracy gives equal weight to all errors
• A model can look better on accuracy and still miss harmful content
• Rare harmful errors may matter more than frequent minor mistakes
• The metric should match the product’s actual objective
• Good metrics reflect the practical cost of different errors

Rubric: Full credit requires explaining that accuracy treats errors uniformly, identifying the harmful-content leak as a high-impact failure, and linking the example to the need for metrics that align with the product’s real priorities.