Learn Before
When F1 is a poor single metric for an imbalanced screening task
Case context: You are building a fraud-detection system for card payments. Fraud cases are rare, but approving legitimate payments quickly is essential. During metric review, one teammate proposes using only the F1 score because it combines precision and recall. Another teammate warns that the system must also measure how well it preserves legitimate transactions.
Question: Given the properties of the F1 score, identify the main risk of using it as the only metric in this fraud-detection setting. Which features of F1 create that risk?
Sample answer: Using only F1 is risky here because F1 does not include true negatives. In a problem where most transactions are legitimate, ignoring true negatives means the metric may hide how well the system handles the large negative class, even though that behavior matters a lot in practice. F1 also gives precision and recall equal weight, which may not reflect the actual business priorities of the fraud screen.
Key points:
- F1 does not account for true negatives.
- Ignoring true negatives can hide poor performance on the dominant negative class.
- F1 gives precision and recall equal importance.
- Equal weighting may not match the needs of the application.
Rubric: The learner should identify that F1 is a poor sole metric in this context because it ignores true negatives, which can leave performance on the majority class unseen and make the evaluation insensitive to the problem's class imbalance.
0
1
Tags
Data Science
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Machine Learning Yearning @ DeepLearning.AI
Related
Which mean is used in the F1 score?
An F1 score of 1.0 means precision and recall are both perfect.
The F1 score is _____ when precision or recall is 0.
Match each F1-related term to the best description.
Order the steps for calculating an F1 score from precision and recall.
Why is the F1 score often used instead of the plain average of precision and recall?
An F1 score can still be misleading when one class is much more common, because it ignores true negatives.
The F1 score is often used in NLP tasks such as named entity recognition and _____ segmentation.
Match each F1 score property to the explanation that best fits it.
When to Use F1 Score
Why the F1 Score Is Useful, and Where It Falls Short
When F1 is a poor single metric for an imbalanced screening task
What is the minimum F1 score, and when does it occur?