Learn Before
Using a Strong Penalty for a Critical Error Type
One way to improve a weak evaluation metric is to assign a very large penalty to a specific mistake that the metric currently treats too lightly, such as allowing unsafe content to pass through.
0
1
Tags
Machine Learning
Deep Learning
Machine Learning Strategy
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Yearning @ DeepLearning.AI
Related
Accuracy Is Not Always the Right Metric
Using a Strong Penalty for a Critical Error Type
Set a New Evaluation Goal When the Current Metric Is Unreliable
What should a team do if its evaluation metric is rewarding the wrong outcome?
If an evaluation metric does not reflect the real project goal, it is still reliable for choosing the best model.
A score that measures the wrong target should not be used to ____ the best model.
Match each metric concept to its description.
What should a team do after noticing its metric points to the wrong goal?
What to do when an evaluation score favors the wrong goal
Diagnose a metric mismatch in a fraud detection model selection process.
What should a team do if its metric does not match the real objective?
What happens when an evaluation metric tracks the wrong goal?
A team should replace an evaluation metric that no longer reflects the project goal.
Learn After
How should a metric change when it fails to penalize a serious mistake?
True or False: Giving a larger penalty to especially serious mistakes can make an evaluation metric better reflect the actual project goal.
To make a metric reflect an unacceptable mistake, you can redesign it to heavily _____ that mistake.
How should an evaluation metric be changed when it misses the real cost of certain mistakes?
A Metric Can Be Adjusted to Reflect the Real Goal
Strengthening a Metric Around a Critical Mistake
Match each term to its definition when revising an evaluation score that rewards the wrong behavior.
Order the steps for correcting a metric that rewards the wrong classifier behavior.
Which example illustrates changing a metric to strongly penalize an unacceptable error?
If a metric misses the real objective, weighting every mistake the same usually fixes it.
Changing a metric to penalize unacceptable mistakes is a way to fix a metric that rewards the wrong project _____.
Match each case to its role in changing a metric so it punishes severe mistakes.
Order the steps for revising a metric to reflect a serious error.
Designing a cost-sensitive metric for high-risk classification errors
Choosing a Metric for a Moderation System with Severe Mistakes
How to Repair a Metric That Misses Critical Errors