Decide whether a team has used the correct criterion for test-set size.
Case context: A team selected a test set but cannot say whether it gives high confidence in the system's overall performance. They argue that choosing any test-set size completes the task.
Question: Diagnose the team's mistake and state what it should require before treating the test set as sufficiently large.
Sample answer: The team has not applied the source's criterion. Merely choosing a size is insufficient; the test set should be large enough to give high confidence in the system's overall performance.
Key points:
- Choosing any size does not establish adequacy.
- Adequacy requires high confidence.
- Confidence must concern overall system performance.
- The test set should be large enough to meet that requirement.
Rubric: The response should identify the missing high-confidence requirement and connect it explicitly to overall system performance and adequate test-set size.
0
1
Tags
Machine Learning
Deep Learning
Machine Learning Strategy
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Yearning @ DeepLearning.AI
Related
What should determine whether a test set is large enough? for Test Set Should Support High-Confidence Performance Estimates
A test set is adequately sized when it supports high confidence in overall system performance.
The test set should give high confidence in the system's _____ performance.
Match each test-set concept to its source-grounded meaning.
Order the reasoning used to judge whether a test set is sufficiently large. for Test Set Should Support High-Confidence Performance Estimates
Explain why test-set size should be judged by confidence in overall performance.
Decide whether a team has used the correct criterion for test-set size.
What is the stated purpose of making the test set large enough?
Which test-set sizing justification best follows the source?
Test-set size should be considered independently of confidence in overall system performance.