When to Use Noise-Smoothing Methods for Learning Curves
Question: In a brief analytical response, explain how to decide whether to average across several subsamples or create class-balanced subsamples when examining learning curves. Include the roles of curve clarity, training-set size, and class balance.
Sample answer: Start by drawing the learning curves themselves. If the lines are already clear enough to reveal the main pattern, there is little reason to add extra smoothing steps. Methods such as averaging over several subsets or building balanced subsets are most helpful when random variation is so large that it hides the underlying trend.
They are also less likely to help when the dataset is fairly large and the class mix is not strongly uneven. For example, with about 18,000 training examples and a near-even split between classes, the curves are often stable enough that these extra steps are unnecessary.
Key points:
- Plot the learning curves before making changes.
- Use extra smoothing only if random variation makes the trend hard to read.
- Averaging over multiple subsets and forming balanced subsets are the relevant techniques.
- A large dataset with only mild class imbalance usually does not justify these extra steps.
Rubric: A strong response says to examine the learning curves first, identifies obscured trends due to excessive noise as the condition that justifies extra processing, names averaging across subsets and constructing balanced subsets as the two methods, and notes that a large dataset with only mild class imbalance makes them less likely to be needed.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
When is it worth considering methods that make learning curves easier to interpret?
It is best to smooth the training and validation curves before looking at them for the first time.
Use noise-reduction methods only when the learning curve is too _____ to show the underlying pattern clearly.
Decide When Noise-Reduction Methods Are Worth Considering
Order the steps for deciding whether to smooth noisy learning curves.
When to Use Noise-Smoothing Methods for Learning Curves
Decide whether additional noise-reducing steps are needed for a learning curve.
Deciding Whether to Average Learning Curves
Which situation least suggests that learning-curve noise reduction is needed?
A very large, fairly balanced training set usually makes these noise-reduction tricks unnecessary.