Google

A common sign of high bias is that, even with the largest available training set, the training error is still much higher than the target performance level. If the training error and development error are close together, the model is not showing much variance; the main problem is that it is underfitting rather than spreading its mistakes between train and dev.

Reading a High-Bias Learning Curve When the Train-Dev Gap Is Small

If a model is still unable to reach the desired error level on its current training set, simply collecting additional labeled examples will not by itself push development error down to that target. As the training set grows, training error does not improve from the extra data alone and may remain unchanged or become slightly worse. In typical cases, development error is also above training error.

More Data Cannot Repair a Model That Still Underfits

Recognizing High Avoidable Bias from Learning Curves

A Small Train-Dev Gap Can Coexist with High Bias

If training error is still far above the best achievable error, the model has _____.

Interpreting Training and Validation Gaps

Steps for identifying strong underfitting from a learning curve

Question:
Describe the usual learning-curve pattern when a model has high avoidable bias. Include what happens to training error, validation error, and the target performance level at the largest training-set size.

Sample answer:
When a model has high avoidable bias, the training error remains much worse than the target performance level even after using the largest training set available. That means the model is still underperforming on the examples it has seen, which is a sign of strong bias. In this pattern, the validation error stays close to the training error, so the gap between them is small. A small train-validation gap suggests the model does not have much variance, even though its bias is high.

Key points:
- Training error stays well above the target performance level
- A large gap between training error and target performance indicates high bias
- The model still performs poorly on the training data
- Training and validation error remain close together
- A small train-validation gap indicates low variance

Rubric:
A full-credit response must explain that, at the largest training-set size, training error is still far from the target performance level and that the train-validation gap is small. The answer should connect the first gap to high bias and the second gap to low variance.

Recognizing High Bias on a Learning Curve

Case context:
You are building a fraud screening system for online payments. Your target error rate for a strong system is 3%. After training on 8,000,000 transactions, you find that the training error levels off at 17%, and the dev error is 18%.

Question:
Using the standard learning-curve pattern, what problem is this model showing, and what do the gaps between 3%, 17%, and 18% mean?

Sample answer:
The model has high avoidable bias. The 14% gap between the target error rate (3%) and the training error (17%) shows that the model is underperforming on the training set, which is a sign of large bias. The 1% gap between the training error (17%) and the dev error (18%) shows that variance is small.

Key points:
- The main issue is high avoidable bias.
- The gap between 3% and 17% indicates large avoidable bias.
- The gap between 17% and 18% indicates small variance.
- The model is not fitting the training data well.

Rubric:
The response must identify high avoidable bias as the primary issue. It must explain that the large gap between target performance and training error reflects bias, and that the small gap between training and dev error reflects low variance.

Diagnosing a Fraud Screening Model's Learning Curve

Question:
In a learning curve that shows high avoidable bias, what does a small difference between training error and development error suggest about the model?

Sample answer:
It suggests the model has low variance.

Key points:
- A small training-dev gap indicates low variance.

Rubric:
Award credit if the learner correctly identifies the small gap as meaning low variance.

Learn Before

Related