Use a Weak Baseline to Find Better Directions
Case context: Your team has built an early churn-prediction model for a subscription app. It performs poorly, and the product manager wants to abandon it and spend many months designing a much more complex system from the ground up.
Question: What should you do with the current baseline model, and what is the main benefit of doing so?
Sample answer: Keep the baseline model and study its behavior instead of discarding it. Looking closely at its mistakes and patterns can reveal where the real problems are, which helps the team choose the most promising improvements much faster than starting over blindly.
Key points:
- Keep the baseline model.
- Analyze its predictions and errors.
- Use that analysis to identify promising improvement directions.
- This avoids wasting time on less useful work.
Rubric: The response should recommend analyzing the baseline model rather than discarding it, and explain that this analysis will expose useful clues for deciding what to improve next.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Why examine early diagnostic signals?
Once a first-pass system is built, looking at its mistakes usually adds little value because the errors do not help identify what to improve next.
Spotting Useful Direction _____
How Practical Clues Help in Machine Learning
Using Early Results to Choose Where to Improve
Why Problem Clues Matter in Machine Learning
Use a Weak Baseline to Find Better Directions
Why a Simple Baseline Is Worth Studying
What Useful Signals Do Real ML Problems Provide?
Strategy Can Save Time