Case Study

How to handle a validation set that is too small to split

Scenario: A research group has a small validation set and wants to keep a separate slice for quick human inspection, but there are not enough examples to carve out another subset. They still need to review mistakes, choose among candidate models, and adjust training settings.

Question: Which data set should serve as the team’s inspection set, and what should it be used for?

Sample answer: Use the full validation set as the inspection set. Apply it to error review, model comparison, and hyperparameter adjustment.

Key points:

  • If the validation set is too small, keep it intact instead of splitting it further.
  • Treat the complete validation set as the inspection set.
  • Use it to study examples the model gets wrong.
  • Use it to choose between models and tune hyperparameters.

Rubric: Full credit requires selecting the entire validation set, identifying its inspection role, and including all three uses. Partial credit applies when the set choice is correct but one or more uses are missing.

0

1

Updated 2026-08-12

Contributors are:

Who are from:

Tags

Machine Learning

Deep Learning

Supervised Learning

Dive into Deep Learning @ D2L

Data Science

Machine Learning Strategy

Machine Learning Yearning @ DeepLearning.AI