Should this team combine two review datasets with a simple model?
Case context: A team is building a sentiment classifier for online product reviews. They currently use hand-crafted word-count features with logistic regression. To add more training examples, they are considering mixing reviews from a mobile app store with reviews from a home-appliance store.
Question: Based on the relationship between algorithm flexibility and the risk of combining datasets, what should the team conclude about their current setup and the proposed data merge?
Sample answer: The team should recognize that their current system is a relatively rigid learning approach: hand-crafted features plus a simple linear classifier. For that kind of model, combining data from two sources with different writing styles and goals can hurt performance rather than help it. So the merge should be treated as risky and tested carefully before adoption. If they were using a much more flexible model, such as a large neural network, the danger from mixing the datasets would be much smaller.
Key points:
- The current model is a less flexible, older-style approach.
- Merging mismatched datasets can degrade performance for that kind of model.
- The risk depends on the model class, not on dataset merging in general.
- A more flexible model would reduce the merging risk.
Rubric: Full credit identifies the current system as a less flexible model and concludes that combining the two review sources may reduce performance for this setup.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Combining Two Image Sources with a High-Capacity Model
Why is it less risky today to combine customer-supplied photos with web-scraped photos in one training set?
True or False: Combining data from different sources could sometimes make older machine-learning systems perform worse.
A classic early vision pipeline used hand-crafted image features followed by a simple _____ classifier.
Match each model type or factor to its role in the risk of combining training data sources.
Order the reasoning steps for deciding whether to combine training datasets.
Why model flexibility changes the risk of combining training data sources
Should this team combine two review datasets with a simple model?
What makes combining training data from multiple sources risky?
Which model family is most exposed to trouble when two data sources disagree?
True or False: Using a very large neural network completely removes the risk of problems when combining multiple training data sources.