Why does a large gap from human performance matter in model debugging?
Question: Answer in one to three sentences.
Sample answer: If a model is much worse than a strong human benchmark, that usually means the system still has substantial headroom. This is useful because it tells the team to investigate several possible fixes, such as improving data quality, changing the model design, or adjusting training settings.
Key points:
- A large gap to human performance suggests substantial room for improvement.
- It helps the team focus on candidate fixes rather than treating the task as fundamentally unsolved.
Rubric: The answer should explain that a large performance gap indicates remaining improvement potential and helps guide the team toward multiple possible remedies.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Human Labeling Works Best for Tasks People Can Do Reliably
Using Human Judgment in Error Review
Use Human-Strong Subsets to Keep Improving
Which set of advantages best explains why comparing with human performance can help ML work?
Comparing a model with strong human performance can help estimate the lowest achievable error and set a realistic target for improvement.
A realistic target error rate can speed up a team’s ____.
Match each reason for comparing against human performance with its use in machine learning work.
Order the reasoning steps for using a human benchmark in model development.
Why comparing algorithm performance to expert performance helps ML teams improve
What to do when a model trails expert performance by a wide margin
Why does a large gap from human performance matter in model debugging?
What choice would most help a team move quickly during model development?
Human-level comparison is useful only for collecting labels.