Learn Before
Use human judgment to interpret model mistakes
Question: Discuss how human intuition can help during error analysis when a machine learning system is performing below human level. Use a speech-recognition example to support your explanation.
Sample answer: When a model is not yet as accurate as a person, error analysis can reveal what clues humans use that the model is missing. For example, if a speech system hears the sentence "I need a new seat by the window" and confuses "seat" with "suite," a person can use the surrounding words and the situation to see which transcript makes sense. That insight helps developers understand which contextual signals matter and adjust the model so it learns to use the same kind of context.
Key points:
- The model is performing below human level.
- Human judgment helps identify useful context.
- The model’s mistakes can be analyzed to find missing cues.
- A speech example should show how context resolves an ambiguity.
- The findings can guide changes to the learning system.
Rubric: A good response states that the model is underperforming relative to humans, explains that human intuition reveals the clues used to choose the correct answer, and applies the idea to a speech-recognition example involving context.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
When is it especially useful to apply human-style error analysis?
A useful aim of error analysis is to infer which clues a person relies on to reach the correct label.
A transcript turned “blueberry muffins” into “_____ muffins.”
Match each part of expert-guided error analysis to its role.
Arrange the steps for human-guided error analysis on a weak model.
How can domain expertise be used when analyzing errors in chest X-ray classification?
Human-guided error analysis is most informative when the model still performs worse than people.
In error analysis, human _____ can help reveal which clues people use to reach the right label.
Connect each case to the way human judgment helps diagnose model mistakes.
Order the steps for using expert intuition to improve a crop-disease image classifier.
Use human judgment to interpret model mistakes
Using clinician insight to improve a chest scan classifier
Using Human Judgment in Error Analysis