Why should an evaluation set contain enough examples?
Question: Answer in one to three sentences.
Sample answer: It should be large enough to give a low-uncertainty estimate of how the model will perform on unseen data. With too few examples, the measured score can vary a lot from chance alone.
Key points:
- Low-uncertainty estimate
- Performance on unseen data
Rubric: Full credit requires both the idea of low uncertainty and the idea of future or unseen performance to be mentioned.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
What makes a test set large enough?
A holdout test set is large enough when it can give a dependable estimate of how the final model will perform.
How Test Set Size Supports Evaluation
Match each test-split idea to its meaning.
Order the steps for deciding whether an evaluation set is large enough.
When Is a Test Set Large Enough?
Judge whether an uncertain evaluation result means the holdout set is too small.
Which statement best reflects how to choose a test-set size?
Test-set size can be chosen without regard to desired confidence.