Explain why small training sets can increase the value of hand-engineered components.
Question: Write a concise analytical response connecting training-set size, human insight, hand-engineered knowledge, and possible system performance.
Sample answer: When the training set is very small, an end-to-end system may not acquire enough knowledge from the available examples. A hand-engineered pipeline can incorporate knowledge derived from human insight. Because the end-to-end system lacks this hand-engineered knowledge, it might perform worse than the pipeline in the small-data setting. Thus, limited training data increases the value of hand-engineered components as a source of algorithmic knowledge.
Key points:
- A very small training set provides limited learned knowledge.
- Human insight supplies much of the needed algorithmic knowledge.
- Hand-engineered components encode that human insight.
- An end-to-end system lacking this knowledge might underperform a hand-engineered pipeline.
Rubric: A strong response accurately explains the small-data condition, identifies human insight as the source of much of the algorithm's knowledge, connects that insight to hand-engineered components, and states that the end-to-end system might—not necessarily will—perform worse.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Why might an end-to-end system underperform a hand-engineered pipeline when training data is scarce?
In a very small-data setting, most algorithmic knowledge may need to come from human insight.
With very little training data, most knowledge may need to come from human _____.
Match each small-data concept with its source-grounded meaning.
Order the reasoning for choosing hand engineering in a small-data setting.
Explain why small training sets can increase the value of hand-engineered components.
Decide how to supply knowledge when an end-to-end system has very little training data.
What supplies most of an algorithm's knowledge when its training set is very small?
Which design response best follows from having a very small training set?
A small training set guarantees that every hand-engineered pipeline will beat every end-to-end system.