Case Study

Diagnosing Bias in a Medical Imaging Classifier

Case context: You built a classifier to detect a rare disease. Your training error is 15%. A panel of expert doctors can only achieve a 14% error rate due to inherent noise in the scans.

Question: Based on this training performance and the expert baseline (optimal error rate), what should you conclude about the classifier's bias, and should you prioritize bias-reducing changes?

Sample answer: The optimal error rate is 14%. Since the classifier's training error is 15%, the avoidable bias gap is only 1%. This tells us that there is very little room for improvement in the classifier's bias. Therefore, we should not prioritize bias-reducing changes.

Key points:

  • Optimal error rate is 14%.
  • The gap between training and optimal error is only 1%.
  • There is little room for improvement in bias.
  • Bias-reducing changes should not be prioritized.

Rubric: Learner correctly identifies the 14% optimal error rate, notes the small 1% gap to the training error, and concludes bias-reducing changes should NOT be prioritized.

0

1

Updated 2026-06-19

Contributors are:

Who are from:

Tags

Machine Learning

Deep Learning

Machine Learning Strategy

Supervised Learning

Dive into Deep Learning @ D2L

Data Science

Machine Learning Yearning @ DeepLearning.AI