High Variance Indicated by 1% Training Error and 5% Training-Dev Error
When humans achieve almost perfect cat-detection performance and the optimal error rate is about 0%, an algorithm with 1% training error, 5% training-dev error, and 5% dev error has high variance.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Related
Use the Best Achievable Human Team to Define Human-Level Performance
Measuring Human Error on Mobile-Image Examples
High Variance Indicated by 1% Training Error and 5% Training-Dev Error
Why is the optimal error rate for cat recognition nearly 0% according to Machine Learning Yearning?
If 14% of audio clips are too noisy for humans to understand, the optimal speech recognition error rate is approximately 14%.
Human-level performance is used as a proxy to estimate the _____ error rate on a given task.
Match each task scenario to the approximate optimal error rate implied by human-level performance.
Order the steps for using human-level performance to estimate optimal error rate and guide bias reduction.
An algorithm achieves 10% error on a task where humans achieve 2% error. What is the avoidable bias and what action does this suggest?
Human-level performance always equals 0% error, so the optimal error rate is always 0% for any machine learning task.
In the cat recognition example, because a human can recognize cats almost all the time, the ideal error rate is nearly _____.
Match each key term to its definition in the context of human-level performance as an optimal error rate proxy.
Order the reasoning steps for deciding whether a task's optimal error rate is near 0% or substantially higher.
Learn After
What primary problem does a model with 1% training error and 5% training-dev error exhibit when optimal error is ~0%?
True or False: When training error is 1%, training-dev error is 5%, and optimal error is ~0%, the algorithm primarily has high bias.
When training error is 1% and training-dev error is 5%, with near-zero optimal error, the algorithm has high _____.
Match each error metric to its role in diagnosing high variance in the cat detection scenario.
Order the steps for diagnosing the primary error type when given training, training-dev, and dev error values.
Which gap in the error analysis directly signals high variance in a model with 1% training error, 5% training-dev error, and 5% dev error?
True or False: In the ML Yearning cat detection example used to illustrate high variance, the dev error and training-dev error are both 5%.
The avoidable _____ is only ~1% in this scenario because training error (1%) is very close to optimal error (~0%).
Match each diagnostic gap to the problem type it reveals in the cat detection error analysis.
Order the reasoning steps that lead to the conclusion 'this model has high variance' given the cat detection error values.