Explaining why a linear classifier stops improving with more data
Case context: You are training a linear classifier to predict equipment failures from sensor readings. Over the last year, your team has collected twelve times as many labeled examples as before. You retrain the model on the much larger dataset and expect a major gain in accuracy, but the score hardly changes.
Question: Using the idea that some learning methods stop benefiting much from extra data, explain why the accuracy did not improve and what this says about the model.
Sample answer: The model has probably reached a performance ceiling. Its learning curve has leveled off, which suggests the linear classifier does not have enough representational capacity to take advantage of much more data. In other words, adding more examples does not help much because the model is already near the limit of what this algorithm can learn from the pattern.
Key points:
- Identify the situation as a plateau or ceiling.
- Explain that the linear classifier has limited capacity.
- State that the learning curve has flattened.
- Conclude that the extra data is not being used effectively by this model.
Rubric: The response should diagnose a plateau and explain that the model's limited capacity, not the amount of training data, is the reason accuracy has barely changed.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
What often happens to an older algorithm as more and more data is added?
Adding more data always guarantees that an older learning algorithm will keep improving.
Older Algorithms and Added Data
Recognizing when a model stops improving
What happens when a legacy model gets more data
Why older models stop gaining from more data
Explaining why a linear classifier stops improving with more data
What Happens to an Older Model with More Data
Which model is most likely to level off early as more labeled data is added?
A stalled learning curve indicates a plateau