Diagnose why a system trained on website images underperforms in the field.
Case context: A team building an image classifier had no way to collect data resembling the images their product would actually encounter in the field. Following common practice, they started by training on a large set of website images. After deployment, the system performs noticeably worse on real field images than it did on their test set.
Question: Based on the source material, what is the most likely explanation for this performance gap, and what does it suggest about the original decision to use website images?
Sample answer: The most likely explanation is the generalization risk explicitly warned about in the source: website images can differ from real future data, so a system trained on them may not generalize well to actual field conditions. This does not necessarily mean the original decision was wrong, since the team had no way to get approximating data, but it confirms the anticipated risk materialized and highlights the need to seek more representative data going forward.
Key points:
- Website images may differ from real future data
- The source predicts exactly this generalization risk
- The original decision may still have been reasonable given no alternatives existed
- The gap signals a need to pursue more representative data
Rubric: Full credit: response connects the performance gap to the generalization risk from the source and acknowledges the decision was reasonable given the lack of alternatives. Partial credit: response identifies the risk but doesn't connect it to the case details. No credit: response invents unrelated causes not grounded in the source.
0
1
Tags
Machine Learning
Deep Learning
Machine Learning Strategy
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Yearning @ DeepLearning.AI
Related
When should a team consider starting with website images for training data?
True or False: Starting with website images carries no risk to system performance.
Using website images risks producing a system that does not _____ well.
Match each data situation to its associated risk or outcome.
Order the reasoning steps for deciding to use website images as a data source.
Analyze the tradeoff involved in using website images when future-like data is unavailable.
Diagnose why a system trained on website images underperforms in the field.
In one to three sentences, explain the risk of using website images for training data.
Which statement best reflects the source's stance on using website images?
True or False: Website images are recommended as the first choice for data collection.