Learn Before
Why can a simple model’s advantage with very little data be hard to predict?
Question: Answer in one to three sentences: Why might a simpler, traditional model sometimes outperform a neural network when labeled data is extremely limited, but not always?
Sample answer: The result depends heavily on how informative the engineered features are. With only a tiny dataset, good feature design can matter more than whether the learner is a logistic regression model, a decision tree, or a neural network.
Key points:
- The outcome depends on the quality of the hand-built features.
- When the dataset is tiny, the specific algorithm family may matter less than the input representation.
Rubric: The response should explain that feature engineering strongly affects performance in the small-data setting and that this can outweigh the choice of model.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
With only about 20 labeled examples, which factor is likely to matter more?
A hand-crafted classifier is always superior to a neural network when the training set is very small.
With about 20 examples, _____ can matter more than algorithm choice.
Match each low-data idea with its meaning.
Arrange the steps for deciding what matters most when training data is extremely limited.
Why feature design can matter more than model choice when data is scarce
Identify the main concern for a team with only 20 labeled examples.
Why can a simple model’s advantage with very little data be hard to predict?
What is the most accurate takeaway about model performance when data size changes?
When a dataset contains only about 20 labeled examples, careful feature design can matter more than choosing between a linear model and a small neural network.