Choose the appropriate target when stakeholders propose a 5% x-ray diagnosis error rate.
Case context: An x-ray diagnosis project has four human baselines: 15% for a typical person with basic training, 10% for a junior doctor, 5% for an experienced doctor, and 2% for a small team that debates each image. Stakeholders propose 5% as the desired error rate.
Question: Diagnose the problem with the proposed target and recommend the human-level proxy and desired error rate supported by the evidence.
Sample answer: The 5% proposal uses the experienced individual doctor rather than the best achievable human baseline. Because the doctor team achieves 2% error, 5% is too high as the human-level proxy and desired performance level. The project should use 2% as its reasonable proxy and desired error rate, while allowing that the true optimal error could be lower.
Key points:
- A team outperforms the experienced individual doctor.
- The proposed 5% target is higher than an achievable 2% result.
- Use 2% as the human-level and optimal-error proxy.
- The actual optimal error could be below 2%.
Rubric: The response should reject 5% using the team evidence, recommend 2% for both the proxy and desired performance, and avoid claiming that 2% must equal the true optimal error.
0
1
References
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Which human baseline should define the optimal-error proxy for the x-ray diagnosis task?
True or false: The optimal error rate cannot be higher than the doctor team's 2% error.
The reasonable human-level performance proxy is _____ error.
Match each x-ray diagnosis baseline to its observed error rate.
Order the reasoning used to select a human-level performance proxy.
Explain why the doctor team's result is a better proxy than an individual doctor's result.
Choose the appropriate target when stakeholders propose a 5% x-ray diagnosis error rate.
Why does a 2% team error place an upper bound on the optimal error rate?
What conclusion is justified by the doctor team's 2% error?
True or false: Using 2% as a proxy means the true optimal error must equal 2%.