Learn Before
Evaluating Training Error as a Measure of Bias
Case context: A machine learning engineer trains a neural network on a very large training dataset and obtains a training set error rate of 15%.
Question: Based on the informal definition in Machine Learning Yearning, what concept does this 15% training error rate represent, and how does the size of the training dataset support this classification?
Sample answer: The 15% error rate represents the algorithm's bias. Under the informal definition, bias is defined as the error rate on the training set. Since the training set is very large, the training error rate is a rough representation of the algorithm's bias.
Key points:
- The 15% error rate on the training set represents the algorithm's bias.
- Bias is roughly the error rate on the training set when that set is very large.
Rubric: The response must identify the 15% training error rate as representing bias under the informal definition, and specify that a very large training set is required for this training error to represent bias.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Related
Adding Training Data Does Not Help Much When Training Error Is High
Classifier with Low Bias and Low Variance Is Doing Well
Negative Avoidable Bias Indicates Training Set Overfitting
Informal Bias and Variance Definitions Differ from Statisticians' Definitions
Training Set Performance Comes Before Dev/Test Performance
What does an algorithm's bias informally measure according to Machine Learning Yearning?
True or False: In Machine Learning Yearning, an algorithm's informal bias is defined as its error rate on the dev/test set.
Informally, an algorithm's _____ is its error rate on the training set when the training set is very large.
What does 'bias' informally refer to according to Machine Learning Yearning?
Bias is informally defined as the algorithm's error rate on the training set.
Informally, an algorithm's _____ is its error rate on the training set.
Match each term to its description in Machine Learning Yearning's bias/variance framework.
Order the steps to correctly estimate an algorithm's bias according to Machine Learning Yearning.
Why does Machine Learning Yearning qualify bias as the training error rate on a 'very large' training set?
In Machine Learning Yearning, bias is defined as the algorithm's error rate on the dev or test set.
Roughly, the bias is the error rate of your algorithm on your _____ set when you have a very large training set.
Match each aspect of the bias definition to what it represents in Machine Learning Yearning.
Order the reasoning steps to determine whether bias is the primary error source in an underperforming ML algorithm.
Explain the Relationship Between Informal Bias and Training Set Size
Evaluating Training Error as a Measure of Bias
Distinguishing the Informal Definition of Bias