Why older models stop gaining from more data
Question: What does it mean when a classic learning method, such as logistic regression, reaches a point where adding more data no longer improves it? Why does that matter?
Sample answer: When a model reaches a plateau, its test performance has essentially stopped getting better even though more training examples are available. On a learning curve, this appears as a flattening line. That is a serious limitation because modern applications often have access to very large datasets, and simple older methods may not have enough representational power to take advantage of all that extra information.
Key points:
- A plateau means performance stops improving with additional data.
- The learning curve becomes flat.
- Logistic regression is one example of a method that can reach this limit.
- Older methods may be unable to use very large datasets effectively.
Rubric: The essay should explain plateauing as the flattening of the learning curve and the end of meaningful improvement from more data. It should also note that older algorithms may not be able to make good use of large modern datasets.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
What often happens to an older algorithm as more and more data is added?
Adding more data always guarantees that an older learning algorithm will keep improving.
Older Algorithms and Added Data
Recognizing when a model stops improving
What happens when a legacy model gets more data
Why older models stop gaining from more data
Explaining why a linear classifier stops improving with more data
What Happens to an Older Model with More Data
Which model is most likely to level off early as more labeled data is added?
A stalled learning curve indicates a plateau