Learn Before
Diagnose a Siamese cat misclassification using error analysis by parts.
Case context: An engineer's Siamese cat classifier pipeline misclassifies an image as not containing a Siamese cat (y=0), even though the correct label is y=1. The engineer wants to know whether to focus further work on the cat detector or the cat breed classifier.
Question: Describe how the engineer should use error analysis by parts to determine which pipeline component is responsible for this specific misclassification, and what she should do with the result.
Sample answer: The engineer should manually examine what each of the two pipeline steps did with this specific image. First she checks whether the cat detector correctly detected and cropped the cat out of the image. Then she checks whether the cat breed classifier correctly classified the crop as a Siamese cat. Based on what she observes, she attributes the error to the cat detector, the breed classifier, or both. She should repeat this process across a sample of misclassified images to see which component is responsible for the largest share of errors, and then prioritize her improvement effort on that component.
Key points:
- Manually examine what the cat detector did for this image
- Manually examine what the breed classifier did for this image
- Attribute the mistake to one or both components
- Repeat across multiple misclassified examples to find the dominant source of error
- Use the attribution to decide where to focus future improvement work
Rubric: Full credit describes manually examining both pipeline steps for the specific image, attributing the error to one or both components, and using the pattern across multiple errors to decide where to focus effort.
0
1
References
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Attributing a Siamese Cat Error to the Cat Detector
Attributing a Siamese Cat Error to the Breed Classifier
What is the purpose of error analysis by parts in the Siamese cat pipeline?
True or False: The Siamese cat pipeline has two parts, a cat detector and a breed classifier.
By carrying out error analysis by _____, you attribute each mistake to one or both pipeline components.
Match each pipeline part to its role in the Siamese cat classifier.
Order the steps for performing error analysis by parts on a misclassified Siamese cat image.
Explain why error analysis by parts is useful before spending years improving a pipeline component.
Diagnose a Siamese cat misclassification using error analysis by parts.
Name the two parts of the Siamese cat classifier pipeline and their roles.
When can a single misclassified image be attributed to both pipeline parts?
True or False: Error analysis by parts requires manually examining what each pipeline step did on a misclassified example.