Learn Before
Irrelevant Training Data and Model Capacity
Question: If a training set contains examples that do not help solve the target task, what harm can they do to a neural network beyond increasing compute time?
Sample answer: They can consume model capacity by pushing the network to allocate parameters to patterns that are not useful for the real task.
Key points:
- They use up model capacity.
- They can cause the network to spend parameters on irrelevant patterns.
- Those patterns may not match the distribution the model will face at evaluation time.
Rubric: The response must state that the irrelevant data uses up the model's capacity.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Why Skip Data That Does Not Match the Evaluation Set?
True or False: If the training, validation, and test sets are drawn from the same distribution, collecting more training examples will always make the model perform better.
According to machine learning strategy, if the dev error curve has _____, adding more training data is unlikely to help you reach your target.
Why might you exclude data that does not add useful information for training?
True or False: More training data always improves validation accuracy.
If the dev error curve has _____, adding more training data is unlikely to move you toward the target performance.
Match each machine-learning concept with the description that fits it best when deciding whether collecting more data is worthwhile.
Order the steps for using a learning curve to judge whether collecting more training data is worthwhile.
In a leaf-disease classifier, why should a large archive of handwritten invoices be left out of training?
True or False: Examining the learning curve can help you avoid spending months collecting more data that later turns out not to improve validation performance.
When compute is limited, examples that add no _____ should be left out of training.
Match each data scenario to the best action.
Order the steps for deciding whether to add a new data source to training.
Why Irrelevant Training Data Should Be Excluded
Whether to Add Contract Scans to a Plant Photo Classifier
Irrelevant Training Data and Model Capacity