Diagnosing Slow Progress in a New Machine Learning Project
Case context: A machine learning team has started a new project. They are trying many different neural network architectures, but they haven't formalized their evaluation criteria or finalized their datasets yet. Consequently, their progress is extremely slow because they keep debating which architecture is actually performing better.
Question: Based on the principles of machine learning project structure, what is the first step this team should take to speed up their workflow, and what specific benefit will this provide?
Sample answer: The team should immediately establish an initial dev/test set and a single evaluation metric. The specific benefit of having these in place is that it will give them an objective way to measure performance, which will help the team iterate quickly instead of arguing over subjective results.
Key points:
- The team currently lacks an objective evaluation mechanism.
- They need to establish an initial dev/test set and metric.
- This configuration will help them resolve debates and iterate quickly.
Rubric: The answer must identify the need to establish the dev/test sets and a metric, and state that this action will help the team iterate quickly.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Primary Benefit of Initial Dev/Test Set and Metric
Dev/Test Set and Metric Impact on Iteration Speed
Benefit of Initial Dev/Test Set and Metric _____
Matching Dev/Test Components to Outcomes
Sequence for Establishing a Quick-Iteration Workflow
Facilitating Quick Iteration in Machine Learning
Rapid Iteration and Initial Setup
Initial Dev/Test Set and Metric _____ Quick Iteration
Matching Quick Iteration Concepts
Decision Path for Achieving Quick Iteration
Analyzing the Iteration Speed Advantage of an Initial Dev/Test Setup
Diagnosing Slow Progress in a New Machine Learning Project
The Core Purpose of Establishing an Early Dev/Test Pipeline