Essay

Why a huge synthetic dataset can still be too narrow

Question: Explain why a training set with 100,000 synthetic images may still lead to overfitting if those images were generated from only about 20 product prototypes. In your response, discuss the difference between the number of samples and the spread of underlying cases.

Sample answer: A large sample count does not automatically mean a broad learning signal. If all 100,000 images come from only about 20 prototypes, then most of the variation is superficial, such as lighting, pose, or rendering details. The model sees many examples, but it is repeatedly learning from the same small set of underlying cases. That gives the training data low effective diversity. By contrast, the development and test sets may include many more real-world product styles. A model trained on a narrow synthetic pool can therefore latch onto quirks of those 20 prototypes instead of learning features that transfer well, which is why it overfits and performs worse on new designs.

Key points:

  • 100,000 images can still reflect only about 20 underlying prototypes
  • The range of underlying cases matters more than the raw image total
  • Development and test data may cover a much wider real-world distribution
  • A mismatch between narrow training variety and broad evaluation variety leads to overfitting

Rubric: Full credit: explains that many images can still come from limited underlying diversity, notes the gap between the training distribution and the development/test distribution, and connects that gap to overfitting. Partial credit: mentions overfitting but does not clearly distinguish sample count from diversity.

0

1

Updated 2026-08-12

Contributors are:

Who are from:

Tags

Machine Learning

Deep Learning

Supervised Learning

Dive into Deep Learning @ D2L

Data Science

Machine Learning Strategy

Machine Learning Yearning @ DeepLearning.AI