Question: Explain, using the definition of Precision, why relying on Precision by itself might give an incomplete picture of a cat classifier's performance on the dev or test set.

Sample answer: Precision only measures the fraction of images labeled as cats that really are cats. A classifier could achieve high Precision by being very conservative and only labeling images as cats when it is extremely confident, even if this means it misses many actual cat images. Because Precision ignores what happens to actual cat images that were not labeled as cats, a classifier can score well on Precision while still performing poorly overall, which is why Machine Learning Yearning pairs it with Recall.

Key points:

• Precision is the fraction of labeled-cat images that really are cats
• Precision ignores actual cat images that were not labeled as cats
• A classifier can achieve high Precision by being overly conservative
• This one-sidedness motivates considering Recall alongside Precision

Rubric: Full credit requires stating the definition of Precision, explaining that it only considers predicted-cat images, and noting that this ignores missed cat images, motivating the need for a complementary metric like Recall.