Diagnosing Bias in a Medical Imaging Classifier
Case context: You built a classifier to detect a rare disease. Your training error is 15%. A panel of expert doctors can only achieve a 14% error rate due to inherent noise in the scans.
Question: Based on this training performance and the expert baseline (optimal error rate), what should you conclude about the classifier's bias, and should you prioritize bias-reducing changes?
Sample answer: The optimal error rate is 14%. Since the classifier's training error is 15%, the avoidable bias gap is only 1%. This tells us that there is very little room for improvement in the classifier's bias. Therefore, we should not prioritize bias-reducing changes.
Key points:
- Optimal error rate is 14%.
- The gap between training and optimal error is only 1%.
- There is little room for improvement in bias.
- Bias-reducing changes should not be prioritized.
Rubric: Learner correctly identifies the 14% optimal error rate, notes the small 1% gap to the training error, and concludes bias-reducing changes should NOT be prioritized.
0
1
Tags
Machine Learning
Deep Learning
Machine Learning Strategy
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Yearning @ DeepLearning.AI
Related
Assessing Bias Reduction Potential from Training Error
Impact of a 14% Optimal Error Rate on a 15% Training Error
High Room for Improvement when Optimal Error is _____
Matching Error Rate Scenarios to Bias Opportunities
Steps to Diagnose Bias using Optimal Error Rate
Analyzing the Relationship Between Optimal Error and Bias
Diagnosing Bias in a Medical Imaging Classifier
Interpreting a 15% Training Error with ~0% Optimal Error
Determining Room for Improvement in Classifier Bias
Evaluating the Fruitfulness of Bias-Reducing Changes