Learn Before
Case Study: Model Improvement Hits a Dead End in Retail Forecasting
Case context: You are building a demand-forecasting model for a chain of grocery stores. Your training data mostly comes from large urban locations, but the dev set focuses on small suburban stores, and the model performs poorly there because the data do not match. You try several ways to simulate suburban shopping patterns, but the results are too unrealistic to be useful. You also cannot collect additional suburban store data because company policy and access restrictions make that impossible.
Question: Using the idea of handling data mismatch, what is the most likely status of your effort to improve the model, and why?
Sample answer: The effort is likely stuck with no clear route to better performance. The main reason is that the team cannot obtain additional training data that truly matches the dev set, and the attempted substitutes do not solve the mismatch. Without a realistic source of matching data, improvement has no dependable path.
Key points:
- Recognize that progress is stalled.
- Explain that the blockage comes from the inability to obtain matching training data.
- Note that the attempted substitutes do not resolve the mismatch.
- State that there is no guaranteed method for overcoming this situation.
Rubric: The response must explain that improvement is blocked because the team cannot get training data that matches the dev set, and it must acknowledge that there is no guaranteed way to fix the mismatch.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
What may happen if you cannot obtain training data that better matches your validation set?
A Mismatch Always Has a Clear Fix
If no additional data can be gathered that better matches the dev set, the route to better performance may not be _____ at first.
Match each data-mismatch idea with the statement that best describes it.
Order the reasoning steps when hard dev examples appear to come from data mismatch.
Why a Lack of Matching Training Data Can Block Improvement
Case Study: Model Improvement Hits a Dead End in Retail Forecasting
Is Success Guaranteed When Seeking Better-Matched Training Data?
What is the most accurate description of handling a training–target data mismatch?
True or False: If you cannot collect training data that better reflects the real-world setting, improving performance may become difficult or stall.