Essay

When Is a Test Set Large Enough?

Question: In a concise analytical response, describe how test-set size, confidence, and final-performance estimation are connected.

Sample answer: A test set is large enough when it supports a dependable estimate of how the final system will perform. The point is not to maximize the number of examples for its own sake. Instead, the test set should be big enough that the estimated performance can be trusted with the desired level of confidence.

Key points:

  • The quantity being estimated is the system’s final performance.
  • The estimate must be reliable enough to inspire confidence.
  • Test-set size is adequate only if it supports that confidence.
  • The needed size depends on the estimation goal, not on a fixed rule.

Rubric: A strong response states that the test set must be big enough to produce a confident estimate of final system performance and makes clear that adequacy is determined by that estimation purpose.

0

1

Updated 2026-08-12

Contributors are:

Who are from:

Tags

Machine Learning

Deep Learning

Supervised Learning

Dive into Deep Learning @ D2L

Data Science

Machine Learning Strategy

Machine Learning Yearning @ DeepLearning.AI