Learn Before
Case Study

When F1 is a poor single metric for an imbalanced screening task

Case context: You are building a fraud-detection system for card payments. Fraud cases are rare, but approving legitimate payments quickly is essential. During metric review, one teammate proposes using only the F1 score because it combines precision and recall. Another teammate warns that the system must also measure how well it preserves legitimate transactions.

Question: Given the properties of the F1 score, identify the main risk of using it as the only metric in this fraud-detection setting. Which features of F1 create that risk?

Sample answer: Using only F1 is risky here because F1 does not include true negatives. In a problem where most transactions are legitimate, ignoring true negatives means the metric may hide how well the system handles the large negative class, even though that behavior matters a lot in practice. F1 also gives precision and recall equal weight, which may not reflect the actual business priorities of the fraud screen.

Key points:

  • F1 does not account for true negatives.
  • Ignoring true negatives can hide poor performance on the dominant negative class.
  • F1 gives precision and recall equal importance.
  • Equal weighting may not match the needs of the application.

Rubric: The learner should identify that F1 is a poor sole metric in this context because it ignores true negatives, which can leave performance on the majority class unseen and make the evaluation insensitive to the problem's class imbalance.

0

1

Updated 2026-08-12

Contributors are:

Who are from:

Tags

Data Science

Machine Learning

Deep Learning

Supervised Learning

Dive into Deep Learning @ D2L

Machine Learning Yearning @ DeepLearning.AI