How should class proportions be chosen for balanced learning-curve samples?
Question: When you create very small subsets to plot a learning curve for data with uneven labels or many classes, what principle should guide how many examples of each class go into a subset?
Sample answer: Make the subset's class mix match the class mix of the full training set as closely as possible.
Key points:
- Do not draw tiny subsets purely at random when the data are highly skewed or have many labels.
- Keep each subset’s label proportions as close as you can to those in the complete training data.
Rubric: The response must say that each class should appear in the subset in roughly the same proportion as in the original training data.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
In a fraud-detection learning curve, why might a class-balanced subsample be better than a purely random subsample when the data are very skewed?
True or False: On highly imbalanced data with many labels, drawing ordinary random subsets usually makes learning curves less noisy than building subsets that preserve the class mix.
When you downsample a dataset with many classes and uneven frequencies, a _____ subset preserves the class mix from the original data as closely as practical.
When is a balanced subset useful for learning-curve plots?
A class-balanced subset for a learning curve should keep class proportions close to those in the full training set.
Reducing curve noise with a representative sample
Match each sampling idea with its description.
Put the steps in order for building one class-balanced sample used to estimate a point on a learning curve.
Why can balanced subsets be useful when drawing learning curves for highly skewed or many-class data?
True or False: Randomly drawing very small training subsets always gives smooth learning curves, even when some classes are rare.
In a 12-example subset that keeps the same 25% positive rate as the full data, you should include _____ positive examples.
Match each data setup to the sampling issue it creates in small learning-curve subsets.
Order the steps for deciding whether to use stratified subsets when drawing learning-curve data.
Why do skewed class distributions make learning curves noisy, and how do balanced subsets help?
Explain why the learning curve is jagged for a skewed classifier and choose an appropriate subset strategy.
How should class proportions be chosen for balanced learning-curve samples?