Learn Before
Changing a Metric to Penalize Unacceptable Errors
One way to change a failing evaluation metric is to heavily penalize letting through pornographic images.
0
1
Tags
Machine Learning
Deep Learning
Machine Learning Strategy
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Yearning @ DeepLearning.AI
Related
Pornographic Image Leakage as a Metric Failure Example
Changing a Metric to Penalize Unacceptable Errors
Choosing a New Trusted Metric Instead of Manual Classifier Selection
What action should a team take when their evaluation metric no longer aligns with the project's actual objectives?
Can a metric that measures the wrong objective still be trusted to select the best machine learning algorithm?
If a metric optimizes the wrong objective, you can no longer trust it to _____ the best algorithm.
Match metric issues and actions to their correct descriptions.
What is the correct sequence of decisions when a team discovers a metric alignment issue?
Analyze the consequences and resolution when a machine learning metric optimizes the wrong objective.
Diagnose a metric alignment issue in a classifier selection process.
What should a team do when their metric cannot be trusted to choose the best model?
Why does measuring the wrong objective invalidate an evaluation metric?
Should a team change their metric if it fails to guide them to the best algorithm?
Learn After
When a metric optimizes the wrong project objective, what is one recommended way to fix it?
True or False: Heavily penalizing unacceptable errors in a metric is a valid way to realign the metric with the true project objective.
To fix a metric that optimizes the wrong objective, you can change the metric to heavily _____ the specific unacceptable error type.
When an evaluation metric optimizes the wrong project objective, what does Ng recommend to fix it?
Heavily penalizing certain error types in an evaluation metric can correct a metric that optimizes the wrong objective.
According to Ng, one way to change a failing evaluation metric is to heavily _____ letting through pornographic images.
Match each term to its definition in the context of fixing a metric that optimizes the wrong objective.
Order the steps for fixing an evaluation metric that optimizes the wrong objective by penalizing unacceptable errors.
What specific example does Ng use in Machine Learning Yearning (p. 25) to illustrate modifying a metric with a heavy penalty?
When a metric fails to optimize the correct objective, treating all error types with equal weight is an adequate solution.
Modifying a metric to penalize unacceptable errors is a technique for correcting a metric that optimizes the wrong project _____.
Match each scenario to the role it plays in Ng's strategy of penalizing unacceptable errors to fix a metric.
Order the reasoning steps a practitioner follows when deciding to add a heavy penalty for a specific error type in an evaluation metric.
Analyzing the Impact of Custom Penalty Weights in Evaluation Metrics for User Safety
Designing a Safe Image Filter Metric for a Public Platform
Mechanism for Correcting an Evaluation Metric that Fails to Penalize Unacceptable Errors