Learn Before
Diagnosing a Fraud Model with 12% Training Error and 27% Dev Error
Case context: A fraud detection model is trained on bank transaction records. It makes mistakes on 12% of the training examples, which you use as an estimate of bias. On the dev set, its error rises to 27%, so the estimated gap due to variance is 15%.
Question: Using standard bias-variance ideas, diagnose this model. Describe what its results on the training set and dev set suggest, and explain why calling it simply overfitting or simply underfitting is not precise.
Sample answer: The model shows both high bias and high variance. Its 12% training error means it is not fitting the training examples well enough, and the jump to 27% on dev data shows it generalizes poorly as well. Because both problems appear together, using only the labels "overfitting" or "underfitting" hides part of the diagnosis.
Key points:
- Identify both high bias and high variance.
- Connect the 12% training error to poor fit on the training set.
- Connect the higher dev error to a generalization gap.
- Note that both issues can occur at the same time, so a single label is incomplete.
Rubric: Evaluates whether the student correctly identifies both high bias and high variance, links each to the reported training and dev errors, and explains why a single overfitting/underfitting label is not fully accurate.
0
1
Tags
Machine Learning
Deep Learning
Machine Learning Strategy
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Yearning @ DeepLearning.AI
Related
A classifier shows about 15% bias and 15% variance. Which description fits best?
True or False: When a classifier shows both high bias and high variance, the usual overfitting/underfitting labels do not cleanly describe the situation.
A classifier with high bias usually performs _____ on the training data.
Identify the Bias and Variance Pattern
Put the diagnostic steps in order for deciding whether a model has high bias, high variance, or both.
In the worked bias–variance diagnosis, what were the two estimated error rates?
True or False: A model that has both high bias and high variance can be overfitting and underfitting at the same time.
Equal Bias and Variance in a Fraud Detector
Match each model-performance pattern to the most likely diagnosis.
Reasoning About Bias and Variance From Training and Validation Results
Explain why a model can show both underfitting and overfitting at the same time.
Diagnosing a Fraud Model with 12% Training Error and 27% Dev Error
What happens when a model has both high bias and high variance?