Learn Before
Pornographic Image Leakage as a Metric Failure Example
In a cat application, classification accuracy may rank classifier A above classifier B even though classifier A occasionally lets pornographic images through. In that situation, classifier A can be more accurate while still being unacceptable for the product.
0
1
Tags
Machine Learning
Deep Learning
Machine Learning Strategy
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Yearning @ DeepLearning.AI
Related
Pornographic Image Leakage as a Metric Failure Example
Changing a Metric to Penalize Unacceptable Errors
Choosing a New Trusted Metric Instead of Manual Classifier Selection
What action should a team take when their evaluation metric no longer aligns with the project's actual objectives?
Can a metric that measures the wrong objective still be trusted to select the best machine learning algorithm?
If a metric optimizes the wrong objective, you can no longer trust it to _____ the best algorithm.
Match metric issues and actions to their correct descriptions.
What is the correct sequence of decisions when a team discovers a metric alignment issue?
Analyze the consequences and resolution when a machine learning metric optimizes the wrong objective.
Diagnose a metric alignment issue in a classifier selection process.
What should a team do when their metric cannot be trusted to choose the best model?
Why does measuring the wrong objective invalidate an evaluation metric?
Should a team change their metric if it fails to guide them to the best algorithm?