Proportionally Stratified Small Subsets Reduce Learning-Curve Noise
For imbalanced or multiclass training data, tiny random subsets can vary sharply in class composition, making learning-curve points noisy. At each subset size, use stratified sampling so the subset approximates the full training set's class proportions; when an exact match is impossible, preserve them as closely as the subset size permits. This reduces variability caused by changing class mix and makes learning-curve points more comparable.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Averaging Errors from Several Small Training Samples
Why can a learning-curve value jump around when it is measured on a tiny random training sample?
A tiny random training sample can make the measured learning-curve error swing noticeably up or down.
A tiny sample with many unclear or incorrect labels is unusually _____.
Match each small-sample condition to its effect on a learning-curve estimate.
Put the checks in a sensible order for explaining a strange value on a tiny sample.
Why can a training curve wobble when the sample size is very small?
Explain a spike caused by a tiny imbalanced sample.
Why can learning-curve values be especially erratic for very small training samples?
Which situation makes a tiny random sample least likely to represent the full dataset?
A very small random sample is less likely to be misleading when the number of classes is large.
Proportionally Stratified Small Subsets Reduce Learning-Curve Noise
Learn After
In a fraud-detection learning curve, why might a class-balanced subsample be better than a purely random subsample when the data are very skewed?
True or False: On highly imbalanced data with many labels, drawing ordinary random subsets usually makes learning curves less noisy than building subsets that preserve the class mix.
When you downsample a dataset with many classes and uneven frequencies, a _____ subset preserves the class mix from the original data as closely as practical.
When is a balanced subset useful for learning-curve plots?
A class-balanced subset for a learning curve should keep class proportions close to those in the full training set.
Reducing curve noise with a representative sample
Match each sampling idea with its description.
Put the steps in order for building one class-balanced sample used to estimate a point on a learning curve.
Why can balanced subsets be useful when drawing learning curves for highly skewed or many-class data?
True or False: Randomly drawing very small training subsets always gives smooth learning curves, even when some classes are rare.
In a 12-example subset that keeps the same 25% positive rate as the full data, you should include _____ positive examples.
Match each data setup to the sampling issue it creates in small learning-curve subsets.
Order the steps for deciding whether to use stratified subsets when drawing learning-curve data.
Why do skewed class distributions make learning curves noisy, and how do balanced subsets help?
Explain why the learning curve is jagged for a skewed classifier and choose an appropriate subset strategy.
How should class proportions be chosen for balanced learning-curve samples?