Changing Architecture Can Shift Both Error Sources
Two models can behave very differently on the same task: one architecture may fit the training data well but generalize poorly, while another may do the opposite. Because of that, changing the architecture can move both training error and dev/test error, not just one of them. Compared with adding more data or enlarging an existing model, trying a new architecture is usually harder to predict.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Architectural Changes Can Lower Bias and Variance Together
Basic Bias and Variance Remedies
Changing Architecture Can Shift Both Error Sources
Why is it useful to diagnose which source of error is larger in an ML project?
Methods that lower bias in a model will always lower its variance too.
Building intuition about _____ and variance helps you decide which change will improve a model.
Match each observation or action to the error issue it mainly helps diagnose or improve.
Determine the Main Source of Error in a Machine Learning Model
Which two error rates are most useful for diagnosing bias and variance?
Reviewing your model's error patterns can help you decide whether to focus on data mismatch.
Bias and variance require different remedies
Match each error pattern to the most likely model issue.
Order the reasoning steps for deciding whether bias or variance is the bigger issue.
Using Error Breakdown to Decide What to Fix First
First Priority for a Fraud Detector with High Underfitting
Using Error Diagnosis to Set the Next Priority
Learn After
How changing model structure can affect error sources
Architecture Changes Are as Predictable as Scaling
Architecture Changes Are Often _____
Ways to Improve a Neural Network
Trying Unusual Model Designs
Why Architecture Changes Can Be Powerful and Risky
Choosing a New Neural Network Design
Why Make a Costly Architecture Change?
Good Places to Find New Model Designs
Are Architecture Changes as Predictable as More Data?