Balanced Small Samples Help Learning Curves Stay Stable on Imbalanced or Multi-Class Data
When the training set is heavily imbalanced or has many classes, very small random subsets can produce erratic learning curves. A better choice is to build each small subset so that the class proportions are close to those in the full training set, which reduces noise in the curve and makes comparisons more reliable.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Averaging Errors from Several Small Training Samples
Balanced Small Samples Help Learning Curves Stay Stable on Imbalanced or Multi-Class Data
Why can a learning-curve value jump around when it is measured on a tiny random training sample?
A tiny random training sample can make the measured learning-curve error swing noticeably up or down.
A tiny sample with many unclear or incorrect labels is unusually _____.
Match each small-sample condition to its effect on a learning-curve estimate.
Put the checks in a sensible order for explaining a strange value on a tiny sample.
Why can a training curve wobble when the sample size is very small?
Explain a spike caused by a tiny imbalanced sample.
Why can learning-curve values be especially erratic for very small training samples?
Which situation makes a tiny random sample least likely to represent the full dataset?
A very small random sample is less likely to be misleading when the number of classes is large.
Learn After
In a fraud-detection learning curve, why might a class-balanced subsample be better than a purely random subsample when the data are very skewed?
True or False: On highly imbalanced data with many labels, drawing ordinary random subsets usually makes learning curves less noisy than building subsets that preserve the class mix.
When you downsample a dataset with many classes and uneven frequencies, a _____ subset preserves the class mix from the original data as closely as practical.
When is a balanced subset useful for learning-curve plots?
A class-balanced subset for a learning curve should keep class proportions close to those in the full training set.
Reducing curve noise with a representative sample
Match each sampling idea with its description.
Put the steps in order for building one class-balanced sample used to estimate a point on a learning curve.
Why can balanced subsets be useful when drawing learning curves for highly skewed or many-class data?
True or False: Randomly drawing very small training subsets always gives smooth learning curves, even when some classes are rare.
In a 12-example subset that keeps the same 25% positive rate as the full data, you should include _____ positive examples.
Match each data setup to the sampling issue it creates in small learning-curve subsets.
Order the steps for deciding whether to use stratified subsets when drawing learning-curve data.
Why do skewed class distributions make learning curves noisy, and how do balanced subsets help?
Explain why the learning curve is jagged for a skewed classifier and choose an appropriate subset strategy.
How should class proportions be chosen for balanced learning-curve samples?