Google

If a model has high bias and is still performing poorly on the training set, reviewing training-set errors can help uncover systematic weaknesses in what the model is learning.

Using Training-Set Mistakes to Diagnose High Bias

When a model performs poorly on the training set, a practical first step is to review about 100 training examples where it makes the largest mistakes. Group those cases into a few error categories to see which kinds of failures are most common.

Sample the Worst Training Cases to Find Error Patterns

When is it useful to analyze errors on the training set?

A model with high bias is usually underfitting.

When bias is high, begin error analysis on the _____ set.

Match each diagnostic idea with its meaning in training analysis.

Order the steps for deciding whether to inspect training-set mistakes.

Question:
Explain why examining mistakes on the training set can still be worthwhile when a model is underfitting.

Sample answer:
If a model performs poorly on the training set, that is a sign of high bias, meaning it is not capturing even the patterns present in the data it has already seen. In that situation, checking which training examples are being missed can help diagnose the source of the problem. This kind of review can use a process similar to the one used for inspecting errors on a small development sample, and it can be done alongside other steps used to reduce bias.

Key points:
- High bias shows up as weak performance on the training set.
- Training-set error review is useful because it focuses on the same data the model is failing to fit.
- The workflow can resemble error inspection on a dev sample.
- It is an extra diagnostic step, not a replacement for other bias-reduction methods.

Rubric:
A strong response states that high bias means the model fits the training set poorly, explains that training-set errors are therefore the right place to look, notes that the review can follow a similar error-analysis procedure used on a dev set, and makes clear that this analysis complements other bias-reduction techniques.

Why Train-Set Error Analysis Matters Under Underfitting

Case context:
A team checks its classifier and finds that it performs badly even on the examples it was trained on. Earlier attempts to reduce simple underfitting have already been tried.

Question:
State the diagnosis and describe the next analysis the team should do, including the style of procedure to follow.

Sample answer:
The model is showing high bias, since it cannot match the training examples well. As a follow-up, the team should perform an error review on the training data itself. A useful way to do that is to use a process like the one used when examining mistakes on a dev set by eye.

Key points:
- Identify the problem as high bias.
- Link that diagnosis to poor fit on the training set.
- Recommend reviewing training examples for errors.
- Use a procedure modeled on manual dev-set error review.

Rubric:
The response should name high bias, advise error analysis on the training set, present it as an additional step after earlier remedies, and mention a protocol similar to manual inspection of dev-set mistakes.

Next Step After a Model Still Fits the Training Set Poorly

Question:
Answer in one to three sentences explaining what high bias means and what kind of error review can help diagnose it.

Sample answer:
High bias means the model is not capturing the pattern in the data it learned from, so its predictions remain weak even on familiar examples. Looking at mistakes made on the training data can help confirm that the model is underfitting.

Key points:
- High bias means the model performs poorly on the data used for learning.
- Checking errors on the training data can help diagnose the problem.

Rubric:
The answer must explain that high bias appears as poor performance on the learning data and that reviewing training-data errors can be useful.

Learn Before

Related