Learn Before
Training Set Performance Comes Before Dev/Test Performance
Dev/test set performance is usually worse than training-set performance. If an algorithm gets 85% accuracy on examples it has seen, there is no way it is getting 95% accuracy on examples it has not seen.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Adding Training Data Does Not Help Much When Training Error Is High
Classifier with Low Bias and Low Variance Is Doing Well
Negative Avoidable Bias Indicates Training Set Overfitting
Informal Bias and Variance Definitions Differ from Statisticians' Definitions
Training Set Performance Comes Before Dev/Test Performance
What does an algorithm's bias informally measure according to Machine Learning Yearning?
True or False: In Machine Learning Yearning, an algorithm's informal bias is defined as its error rate on the dev/test set.
Informally, an algorithm's _____ is its error rate on the training set when the training set is very large.
What does 'bias' informally refer to according to Machine Learning Yearning?
Bias is informally defined as the algorithm's error rate on the training set.
Informally, an algorithm's _____ is its error rate on the training set.
Match each term to its description in Machine Learning Yearning's bias/variance framework.
Order the steps to correctly estimate an algorithm's bias according to Machine Learning Yearning.
Why does Machine Learning Yearning qualify bias as the training error rate on a 'very large' training set?
In Machine Learning Yearning, bias is defined as the algorithm's error rate on the dev or test set.
Roughly, the bias is the error rate of your algorithm on your _____ set when you have a very large training set.
Match each aspect of the bias definition to what it represents in Machine Learning Yearning.
Order the reasoning steps to determine whether bias is the primary error source in an underperforming ML algorithm.
Explain the Relationship Between Informal Bias and Training Set Size
Evaluating Training Error as a Measure of Bias
Distinguishing the Informal Definition of Bias
Learn After
What must occur before strong dev/test performance can reasonably be expected?
Dev/test performance is usually worse than training-set performance.
An algorithm must perform well on the _____ set before strong dev/test performance can be expected.
Match each performance concept to its source-based meaning.
Order the reasoning used to evaluate a training-versus-dev/test performance claim.
Explain why training-set performance sets a prerequisite for dev/test performance.
Diagnose a report claiming 85% training accuracy and 95% dev/test accuracy.
Why should training results be checked before expecting strong dev/test results?
Which reported result conflicts with the source's training-first principle?
An algorithm performing poorly on training data can still be expected to perform well on dev/test data.