Learn Before
Why Irrelevant Training Data Should Be Excluded
Question: Explain why adding training examples that do not help with the target task can hurt a learning system. In your answer, discuss both training cost and the model's limited ability to store useful patterns, especially when those examples fall outside the development and test distribution.
Sample answer: Training on examples that do not support the target task increases the amount of computation required, because the optimizer must still process them during training. Those examples also consume part of the model's limited capacity, since the network uses parameters to fit patterns that are not useful for the distribution the system is expected to handle. For that reason, data far outside the development and test distribution should generally be left out unless it has some clear benefit.
Key points:
- Unhelpful examples increase training compute.
- They also use up model capacity that could be devoted to useful patterns.
- Learning from data outside the target distribution can pull the model toward irrelevant features.
- Excluding such data is reasonable when it provides no expected benefit.
Rubric: A satisfactory response must explain that unhelpful training data increases computational cost during training and also uses limited model capacity on patterns that do not help with the development/test distribution.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Why Skip Data That Does Not Match the Evaluation Set?
True or False: If the training, validation, and test sets are drawn from the same distribution, collecting more training examples will always make the model perform better.
According to machine learning strategy, if the dev error curve has _____, adding more training data is unlikely to help you reach your target.
Why might you exclude data that does not add useful information for training?
True or False: More training data always improves validation accuracy.
If the dev error curve has _____, adding more training data is unlikely to move you toward the target performance.
Match each machine-learning concept with the description that fits it best when deciding whether collecting more data is worthwhile.
Order the steps for using a learning curve to judge whether collecting more training data is worthwhile.
In a leaf-disease classifier, why should a large archive of handwritten invoices be left out of training?
True or False: Examining the learning curve can help you avoid spending months collecting more data that later turns out not to improve validation performance.
When compute is limited, examples that add no _____ should be left out of training.
Match each data scenario to the best action.
Order the steps for deciding whether to add a new data source to training.
Why Irrelevant Training Data Should Be Excluded
Whether to Add Contract Scans to a Plant Photo Classifier
Irrelevant Training Data and Model Capacity