Essay

How should holdout splits change when a dataset becomes very large?

Question: Describe why a fixed 70/30 train/test split is a poor choice for very large machine learning datasets. Compare a small dataset with a few hundred or a few thousand records to a massive dataset with tens of millions or more records. In your explanation, focus on how much data should be reserved for development and final evaluation, and why the role of those sets matters more than keeping a fixed percentage.

Sample answer: A 70/30 split can be practical for smaller projects because leaving out 30% still gives a training set that is large enough, while also providing enough holdout examples to judge performance. For very large datasets, though, keeping 30% aside would discard an enormous amount of useful training data without adding much evaluation value. The purpose of the development and test sets is to estimate how well a model generalizes and to compare model choices, not to consume a fixed share of the data. As the total dataset grows, the holdout fraction should become smaller, even though the actual number of examples in those sets can stay the same or increase a little so the evaluation remains reliable.

Key points:

  • A 70/30 split can be reasonable when the dataset is small or moderate in size.
  • With very large datasets, reserving 30% for holdout use wastes too many training examples.
  • The share assigned to development and test sets should shrink as the dataset gets bigger.
  • What matters is having enough holdout examples to measure performance accurately, not preserving a particular percentage.

Rubric: The response must explain: 1. Why a 70/30 split can work for smaller datasets. 2. Why keeping that same percentage is inefficient for very large datasets. 3. That development and test sets are meant to evaluate and compare models, so they only need enough examples to do that well, rather than a fixed large fraction of the data.

0

1

Updated 2026-08-12

Contributors are:

Who are from:

Tags

Machine Learning

Deep Learning

Supervised Learning

Dive into Deep Learning @ D2L

Data Science

Machine Learning Strategy

Machine Learning Yearning @ DeepLearning.AI