Learn Before
Using clinician insight to improve a chest scan classifier
Case context: A hospital team has built a model that flags a condition on chest scans, but its performance is still worse than that of experienced radiologists.
Question: What is the best way to use the radiologists' knowledge to diagnose the model's weaknesses and improve it, and what specific information are you trying to learn from them?
Sample answer: The team should review the model's mistakes with the radiologists and ask them to explain how they reached the correct conclusion on each case. The main goal is to find out which visual cues, patterns, or clinical context the experts relied on so the model can be adjusted to pay attention to the same kinds of signals.
Key points:
- Review misclassified cases directly with radiologists.
- Ask them to explain their reasoning.
- Identify the cues or context they use.
- Use that insight to guide changes to the model.
Rubric: A good answer should describe discussing the failed cases with experts and should specify that the team is trying to extract the cues and reasoning the experts used to make the correct call.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
When is it especially useful to apply human-style error analysis?
A useful aim of error analysis is to infer which clues a person relies on to reach the correct label.
A transcript turned “blueberry muffins” into “_____ muffins.”
Match each part of expert-guided error analysis to its role.
Arrange the steps for human-guided error analysis on a weak model.
How can domain expertise be used when analyzing errors in chest X-ray classification?
Human-guided error analysis is most informative when the model still performs worse than people.
In error analysis, human _____ can help reveal which clues people use to reach the right label.
Connect each case to the way human judgment helps diagnose model mistakes.
Order the steps for using expert intuition to improve a crop-disease image classifier.
Use human judgment to interpret model mistakes
Using clinician insight to improve a chest scan classifier
Using Human Judgment in Error Analysis