Learn Before
What should a team do if its metric does not match the real objective?
Question: If the score used for model selection rewards the wrong behavior, how should the team treat that score, and what is the next step?
Sample answer: The score should no longer be treated as a dependable way to choose among models. The team should redefine the evaluation measure so it reflects the actual project goal.
Key points:
- The current score is not dependable for selecting the best model.
- The evaluation measure should be changed to match the real objective.
Rubric: The answer should say that the metric is not trustworthy for model selection and that the team needs to change the evaluation metric.
0
1
Tags
Machine Learning
Deep Learning
Machine Learning Strategy
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Yearning @ DeepLearning.AI
Related
Accuracy Is Not Always the Right Metric
Using a Strong Penalty for a Critical Error Type
Set a New Evaluation Goal When the Current Metric Is Unreliable
What should a team do if its evaluation metric is rewarding the wrong outcome?
If an evaluation metric does not reflect the real project goal, it is still reliable for choosing the best model.
A score that measures the wrong target should not be used to ____ the best model.
Match each metric concept to its description.
What should a team do after noticing its metric points to the wrong goal?
What to do when an evaluation score favors the wrong goal
Diagnose a metric mismatch in a fraud detection model selection process.
What should a team do if its metric does not match the real objective?
What happens when an evaluation metric tracks the wrong goal?
A team should replace an evaluation metric that no longer reflects the project goal.