Google

A simple way to respond to these errors is to make the model more flexible when it is underfitting, and to use more training data when it is overfitting.

Basic Bias and Variance Remedies

When a system is underfitting, adding capacity can help it learn the training patterns more accurately. In a neural network, this usually means using more layers, more hidden units, or both. A larger model may also increase variance, so techniques such as weight decay, dropout, or early stopping are often used to keep that side effect under control.

Why enlarging a model can lower bias

When a model shows high variance, adding more labeled examples is often the most dependable fix, provided additional data is available and the system can be trained on it. In many cases, more training data lowers variance without changing bias much.

Using More Data When Variance Is High

In theory, making a neural network larger and giving it more training examples can improve performance on many tasks. In real projects, however, very large models take much longer to train, and the pool of additional data can run out.

Why Increasing Model Size and Data Eventually Stops Helping

If a model is showing high avoidable bias, several practical fixes can help: increase model capacity, revise the input features using error analysis, reduce or remove regularization, or adjust the architecture so it better matches the task. Simply adding more training examples is usually not the right response for this problem, because extra data often improves variance more than bias.

Methods to Reduce High Avoidable Bias

When a model has high variance, common remedies include collecting more training examples, adding regularization, using early stopping, simplifying the feature set, or changing the model so it is less flexible. Another option is to adjust the input representation or architecture if error analysis shows that the current setup is too sensitive to the training data.

Reducing model size can also lower variance, but it should be treated carefully. If computation is not the main limitation, regularization often gives better overall classification performance than simply making the network smaller.

Ways to Reduce High Variance

What should you do when avoidable bias is high?

True or false: When a model has high variance, adding more training examples is often a useful remedy.

For persistent underfitting, increase the _____ of the model.

Match each learning problem or model change with the usual remedy or meaning.

Order the steps for applying a simple model-fix rule.

Question:
In a short analytical response, explain how a simple bias/variance diagnosis leads to different remedies for each problem.

Sample answer:
The diagnosis points to different actions depending on the source of the error. If a model has high avoidable bias, a common response is to make the model more expressive, for example by adding more layers or hidden units to a neural network. If the model has high variance, a common response is to expand the training set so the model sees more examples. In other words, the remedy depends on whether the main issue is bias or variance.

Key points:
- High avoidable bias is addressed by increasing model capacity.
- Adding layers or hidden units is one way to increase neural-network capacity.
- High variance is addressed by using more training data.
- The appropriate fix depends on which problem is diagnosed.

Rubric:
A strong response correctly matches each problem with its remedy, gives a valid example of increasing model capacity, and clearly states that the choice of action depends on the diagnosis.

How the diagnosis determines the fix for bias and variance

Case context:
Project North has been diagnosed with high avoidable bias in its neural network.
Project South has been diagnosed with high variance in its system.

Question:
Using the simple remediation formula, what should each project do, and why?

Sample answer:
Project North should increase model capacity, for example by adding layers or hidden units, because the diagnosis is high avoidable bias. Project South should collect more training examples because the diagnosis is high variance.

Key points:
- Project North should use a larger model.
- Adding layers or hidden units is a valid example for Project North.
- Project South should add more training data.
- Each recommendation must match the diagnosed problem.

Rubric:
The response should assign a larger model to Project North and more training data to Project South, with each recommendation tied to the correct diagnosis.

Select the right remedy for bias and variance in two projects.

Question:
In one to three sentences, state the usual response to a model with high avoidable bias and the usual response to a model with high variance.

Sample answer:
For high avoidable bias, increase the model’s capacity or reduce constraints that make it too limited. For high variance, use more training data or add regularization.

Key points:
- High avoidable bias: increase model capacity.
- High variance: add training data or regularization.

Rubric:
The answer must identify both actions correctly and must not swap them.

Learn Before

Related