Why should an evaluation set contain enough examples?
Question: Answer in one to three sentences.
Sample answer: It should be large enough to give a low-uncertainty estimate of how the model will perform on unseen data. With too few examples, the measured score can vary a lot from chance alone.
Key points:
- Low-uncertainty estimate
- Performance on unseen data
Rubric: Full credit requires both the idea of low uncertainty and the idea of future or unseen performance to be mentioned.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
What makes a test set large enough?
A holdout test set is large enough when it can give a dependable estimate of how the final model will perform.
How Test Set Size Supports Evaluation
Match each test-split idea to its meaning.
Order the steps for deciding whether an evaluation set is large enough.
When Is a Test Set Large Enough?
Judge whether an uncertain evaluation result means the holdout set is too small.
Why should an evaluation set contain enough examples?
Which statement best reflects how to choose a test-set size?
Test-set size can be chosen without regard to desired confidence.