Learn Before
Trade-offs of Decreasing Model Size for Variance Control
Question: Under what condition is decreasing a model's size justified as a method to manage variance, and what is the primary disadvantage of this approach compared to regularization?
Sample answer: Decreasing model size is justified when speeding up training and reducing computational cost is a priority. Its primary disadvantage compared to regularization is that it can increase bias and typically yields worse overall classification performance.
Key points:
- Decreasing model size is justified if the goal is to speed up training and reduce computational cost.
- Decreasing model size can increase bias and yields inferior classification performance compared to regularization.
Rubric: The student must state that decreasing model size is justified for faster training/reduced computational cost, and that its primary drawback is poorer classification performance/increased bias compared to regularization.
0
1
References
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
When the goal is reducing variance and computational cost is not a concern, which technique does Andrew Ng recommend over decreasing model size?
Andrew Ng recommends decreasing model size as the go-to technique for reducing variance in Machine Learning Yearning.
Adding _____ usually gives better classification performance than reducing model size when addressing variance.
Match each model-size-reduction concept to its correct description from ML Yearning.
Order the decision steps for responding to a high-variance problem per Andrew Ng's guidance in ML Yearning.
According to ML Yearning, what is the main reason you might choose to decrease model size even though it is not the best variance remedy?
Decreasing model size can decrease variance while also potentially increasing bias, according to ML Yearning.
Reducing model size is most justified when _____ model training is the priority, rather than purely reducing variance.
Match each scenario or goal to the appropriate technique or outcome Andrew Ng describes in ML Yearning.
Order the reasoning steps Andrew Ng applies when evaluating whether to reduce model size as a variance remedy.
Evaluating Model Size Reduction vs Regularization for Variance Reduction
Optimizing a High-Variance Speech Recognition System under Training Constraints
Trade-offs of Decreasing Model Size for Variance Control