Choosing a Fast Baseline for a Ticket Routing Project
Case context: A product team is starting a new support-ticket routing system for a software company. One engineer wants to classify tickets using keyword counts, another wants to rely on sender department metadata, and a third wants to build a more elaborate model that combines message embeddings with account history. The team is unsure which direction deserves the most effort.
Question: Using the idea of rapid prototyping, what should the team do first to settle the disagreement and learn which approach is most promising?
Sample answer: The team should avoid spending a long time debating the perfect design before seeing evidence. A better first step is to build a simple routing model quickly, even if it is clearly incomplete or inaccurate in some cases. After that baseline is running, the team can inspect its errors to learn where it fails and which improvements are most likely to matter. That evidence helps the team choose among the competing ideas instead of guessing.
Key points:
- Do not spend a long time arguing for an ideal design before testing
- Build a simple working baseline quickly, even if it is not very strong
- Use error analysis on the baseline to identify the most promising next step
Rubric: The answer should recommend constructing a basic, likely imperfect system quickly rather than designing the final solution first, and it should explain that examining the baseline's errors will help decide which direction to pursue.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Starting Direction in a New Spam-Filtering Project
Advice for Building a First Working AI System
What should a team do first when starting an ML project in a domain it does not know well?
Should you wait to design a perfect machine learning system before building any baseline at all?
Once a baseline model is available, which diagnostic review helps you find recurring failure patterns?
Why is it usually better to start with a simple baseline system before investing in a highly elaborate design?
Error analysis can be done effectively before any first model is built, because it only requires labeled data rather than model predictions.
A practical strategy is to build and train a simple first system quickly, perhaps in just a _____ days.
Match each step in a fast-iteration development process to its purpose.
Order the recommended startup workflow for a new machine learning project
What is the main advantage of building a simple first model before a more sophisticated one?
Can an expert usually identify the best ML approach for a new problem without trying any prototypes?
After building a simple baseline system, the recommended next step is _____ to find the most useful improvements and guide iteration.
Match each machine learning recommendation to the reason it is useful.
Order the steps that support a fast baseline-and-iterate approach.
Why a Strong First Draft Beats Trying to Perfect the Plan
Choosing a Fast Baseline for a Ticket Routing Project
Why Study a Weak Baseline System?