Analyzing the Pitfalls of Building a Perfect System at the Outset
Question: Explain why trying to design and build a perfect machine learning system at the start of a project in a new domain is typically counterproductive. Use the concepts of domain expertise, predicting project direction, and error analysis to support your analysis.
Sample answer: Designing a perfect system from the start is difficult because when beginning a project, especially in a new domain where one is not an expert, it is hard to correctly guess the most promising directions. Instead of trying to build the perfect system, it is more effective to quickly build and train a basic system (potentially in a few days). Examining this basic system, even if it is suboptimal, provides crucial clues. By running error analysis on its output, you can identify the most promising directions and iteratively improve the algorithm.
Key points:
- Difficulty in guessing the most promising directions in a new domain
- Avoiding the trap of trying to design/build the perfect system at the start
- Building and training a basic system quickly (within a few days)
- Using error analysis to guide iterative improvement
Rubric: The response should address: 1) the difficulty of guessing promising directions in a new domain, 2) the risk of wasting time building a complex but incorrect system at the outset, and 3) how error analysis on a basic system provides data-driven directions for iteration.
0
1
References
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Related
Email Anti-Spam System as a New System Example
Quick Basic-System Advice Targets AI Applications
What is the recommended first step when starting a new ML project in an unfamiliar area?
True or False: You should spend significant time designing the perfect ML system before building or training anything.
After building a basic ML system quickly, you should use _____ to identify the most promising directions for iterative improvement.
Why does Machine Learning Yearning recommend building a basic system quickly rather than designing a perfect system at the outset?
According to Machine Learning Yearning, error analysis should be performed before building any initial system in order to avoid wasted effort.
Machine Learning Yearning recommends building and training a basic system as quickly as possible—perhaps in just a _____ days.
Match each phase of the quick-iteration workflow to its purpose as described in Machine Learning Yearning.
Order the steps of Machine Learning Yearning's recommended workflow for starting a new ML project from beginning through iteration.
What key value does Machine Learning Yearning say you gain by examining a basic system, even when it is far from the best system you could build?
Machine Learning Yearning states that experienced ML practitioners can reliably identify the best project direction before building any system.
After building a basic system, Machine Learning Yearning recommends using _____ to identify the most promising directions and iteratively improve the algorithm.
Match each Machine Learning Yearning recommendation to the reasoning the book provides for it.
Order the reasoning steps that justify the quick-build-and-iterate strategy recommended in Machine Learning Yearning.
Analyzing the Pitfalls of Building a Perfect System at the Outset
Resolving Strategy Disagreements in a New Anti-Spam Project
The Purpose of Examining a Suboptimal Basic System