Learn Before
Choosing One Score for a Risk-Flagging Model
Case context: You are building a message filter for an online marketplace that flags seller messages as risky. Missing a harmful message is bad, but incorrectly flagging a legitimate message also creates problems. Several candidate models are available, and some score well on one measure but poorly on the other.
Question: What evaluation approach should the team use to make model comparison easier, and what is a simple way to turn the two measures into one number?
Sample answer: Use a single combined evaluation score rather than comparing the two measures separately. A simple way to build that score is to compute the arithmetic mean of precision and recall for each model.
Key points:
- Recommend merging precision and recall into one evaluation value.
- Explain that two separate scores can make model selection confusing.
- Use the average of precision and recall as a straightforward computation method.
Rubric: The response must state that precision and recall should be combined into one number to compare models more easily, and it must identify averaging them as a valid way to compute that number.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
A common way to combine precision and recall
True or False: Precision and recall are commonly combined by taking their simple arithmetic mean.
A common single-number summary that balances precision and recall is the _____ mean of the two.
Why combine precision and recall into one score when both matter?
Averaging precision and recall
One Simple Way to Merge Precision and Recall
Match the machine-learning metric to its description.
Order the steps for turning precision and recall into one decision metric.
Which simple operation can combine precision and recall into one score?
If precision and recall are both important, the best practice is to keep them as separate metrics instead of combining them into one score.
When precision and recall both matter, use a _____ metric.
Match each idea to the corresponding evaluation takeaway.
Put the steps in order for replacing two separate evaluation metrics with one overall score.
How should precision and recall be evaluated together?
Choosing One Score for a Risk-Flagging Model
One Simple Way to Combine Precision and Recall