Use the Best Achievable Human Team to Define Human-Level Performance
When several human baselines are available, the best achievable human performance can be used as the human-level proxy for the optimal error rate. In the medical-imaging example, a team of doctors achieves 2% error, so 2% is a reasonable desired performance level and optimal-error proxy; 5% or 10% would be too high because a team can do better.
0
1
References
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Related
Use the Best Achievable Human Team to Define Human-Level Performance
Measuring Human Error on Mobile-Image Examples
High Variance Indicated by 1% Training Error and 5% Training-Dev Error
Why is the optimal error rate for cat recognition nearly 0% according to Machine Learning Yearning?
If 14% of audio clips are too noisy for humans to understand, the optimal speech recognition error rate is approximately 14%.
Human-level performance is used as a proxy to estimate the _____ error rate on a given task.
Match each task scenario to the approximate optimal error rate implied by human-level performance.
Order the steps for using human-level performance to estimate optimal error rate and guide bias reduction.
An algorithm achieves 10% error on a task where humans achieve 2% error. What is the avoidable bias and what action does this suggest?
Human-level performance always equals 0% error, so the optimal error rate is always 0% for any machine learning task.
In the cat recognition example, because a human can recognize cats almost all the time, the ideal error rate is nearly _____.
Match each key term to its definition in the context of human-level performance as an optimal error rate proxy.
Order the reasoning steps for deciding whether a task's optimal error rate is near 0% or substantially higher.