Learn Before
Pornographic Image Leakage as a Metric Failure Example
In a cat application, classification accuracy may rank classifier A above classifier B even though classifier A occasionally lets pornographic images through. In that situation, classifier A can be more accurate while still being unacceptable for the product.
0
1
Tags
Machine Learning
Deep Learning
Machine Learning Strategy
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Yearning @ DeepLearning.AI
Related
Pornographic Image Leakage as a Metric Failure Example
Changing a Metric to Penalize Unacceptable Errors
Choosing a New Trusted Metric Instead of Manual Classifier Selection
What action should a team take when their evaluation metric no longer aligns with the project's actual objectives?
Can a metric that measures the wrong objective still be trusted to select the best machine learning algorithm?
If a metric optimizes the wrong objective, you can no longer trust it to _____ the best algorithm.
Match metric issues and actions to their correct descriptions.
What is the correct sequence of decisions when a team discovers a metric alignment issue?
Analyze the consequences and resolution when a machine learning metric optimizes the wrong objective.
Diagnose a metric alignment issue in a classifier selection process.
What should a team do when their metric cannot be trusted to choose the best model?
Why does measuring the wrong objective invalidate an evaluation metric?
Should a team change their metric if it fails to guide them to the best algorithm?
Learn After
In the cat app example, why is classifier A ultimately unacceptable despite higher accuracy?
True or False: A classifier with higher classification accuracy is always the better choice for a product.
Classifier A is more accurate but occasionally lets _____ images slip through, making it unacceptable.
Match each classifier in the cat application example to its corresponding property.
Order the steps of reasoning that reveal classification accuracy is the wrong metric in the cat app example.
Explain why classification accuracy failed to capture the true product objective in the cat application example.
A cat app team must decide between two classifiers after accuracy testing reveals a hidden flaw.
In one to three sentences, describe the specific failure that makes classifier A unacceptable in the cat app example.
What does the pornographic image leakage example primarily illustrate about metrics?
True or False: Classifier B is described as also occasionally letting pornographic images slip through.