Learn Before
Applying early stopping to mitigate model overfitting.
Case context: You are training a neural network and notice that while the training error continues to decrease, the development set error has started to increase. To address this, you implement a method to halt the optimization algorithm based on the dev-set error.
Question: Based on Machine Learning Yearning, what technique are you applying, what optimization process is being halted, and what specific trade-off between bias and variance should you expect?
Sample answer: You are applying early stopping, which means you stop gradient descent early based on the dev-set error. The expected trade-off is that this technique will reduce the model's variance but it will increase the bias.
Key points:
- The technique is early stopping
- It halts gradient descent
- It reduces variance
- It increases bias
Rubric: The learner must correctly identify the technique as early stopping, note that gradient descent is the process being stopped, and explicitly state that it reduces variance while increasing bias.
0
1
Tags
Data Science
D2L
Dive into Deep Learning @ D2L
Machine Learning
Deep Learning
Supervised Learning
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
According to Machine Learning Yearning, on which metric is early stopping based?
True or False: Early stopping reduces variance but increases bias in a trained model.
Early stopping means stopping _____ early, based on dev set error.
Match each concept to its specific role in the early stopping technique.
Order the steps for implementing early stopping during model training.
How do some ML authors classify early stopping, according to Machine Learning Yearning?
True or False: Early stopping halts gradient descent based on training set error, not dev set error.
Early stopping behaves a lot like _____ methods, which is why some authors call it a regularization technique.
Match each description to the concept it best characterizes in the context of early stopping.
Order the reasoning steps for deciding whether to use early stopping to address a model's overfitting.
Analyze the trade-offs of early stopping in gradient descent.
Applying early stopping to mitigate model overfitting.
Explain the mechanism and primary trade-off of early stopping.