Learn Before
How to check whether synthetic training data looks unrepresentative
Question: In one to three sentences, describe a simple test for whether synthetic training examples are likely not representative.
Sample answer: Train a classifier to separate synthetic examples from real ones. If it can do that easily, the synthetic set probably reflects a narrow source rather than the full real-world distribution—for example, images generated from only 15 product sketches or audio created from just 30 seconds of recording.
Key points:
- A useful test is whether real and synthetic examples are easy to distinguish.
- Easy separability suggests the synthetic set came from a narrow source.
- A narrow source might be a small set of sketches or a very short recording.
Rubric: Full credit: states the separability test and gives at least one concrete narrow-source example. Partial credit: mentions that synthetic data may be unrepresentative without explaining how to test it.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Overfitting from Reused Background Noise
Synthetic Data Can Still Cause Narrow Memorization
What is the main concern when synthetic examples are easy to tell apart from real data?
Representativeness of Synthetic Noise Samples
Synthetic Data Should Reflect the Real Task
Match each synthetic-data case to whether it is representative.
Order the steps for checking whether generated examples are truly representative.
Why synthetic training examples must reflect real variation
Check a Synthesized Audio Data Plan for Hidden Artifacts
How to check whether synthetic training data looks unrepresentative
Which synthetic-data strategy is least likely to create a representativeness problem?
True or False: Synthetic data built from only a small set of source examples can be unrepresentative.