Case Study

Explain a spike caused by a tiny imbalanced sample.

Case context: A fraud-detection dataset has far fewer fraud cases than ordinary transactions. When training error is plotted for random subset sizes, the point at 12 examples is noticeably unusual. Looking at that 12-example draw shows that every record is a normal transaction.

Question: What is the most likely explanation for the unusual point, and why would this particular sample be a poor basis for learning?

Sample answer: The best diagnosis is that the 12-example sample is not representative because the class mix is heavily unbalanced and the random draw happened to miss every fraud case. Since the sample contains only normal transactions, the model gets no chance to observe the distinguishing pattern it needs to separate the two classes, so the measured error at that point can look erratic.

Key points:

  • The sample does not reflect the full data distribution
  • Severe class imbalance makes this outcome plausible
  • The 12 records include no fraud examples
  • Without fraud cases, the learner cannot practice the target separation
  • The strange curve value can come from random sampling variation

Rubric: Full credit requires stating that class imbalance plus random sampling produced an unrepresentative subset, noting that no fraud examples were included, and explaining that this leaves the learner unable to learn the class distinction and can create a noisy learning-curve value.

0

1

Updated 2026-08-12

Contributors are:

Who are from:

Tags

Machine Learning

Deep Learning

Supervised Learning

Dive into Deep Learning @ D2L

Data Science

Machine Learning Strategy

Machine Learning Yearning @ DeepLearning.AI