Learn Before
Diagnosing a Self-Driving Car Pipeline
Case context: You are leading an ML team building a self-driving car. Through rigorous testing, you verify that your object detection, lane tracking, and path planning components all perform at human-level when evaluated individually. However, the overall self-driving car system plans significantly worse paths than a human given the same camera images.
Question: Based on Andrew Ng's framework, diagnose the fundamental problem with your self-driving car system and state the necessary decision your team must make.
Sample answer: The fundamental problem is that the ML pipeline itself is flawed. Because the system underperforms a human despite having human-level components, the overall architecture is insufficient. The team's necessary decision is to redesign the ML pipeline entirely.
Key points:
- Diagnose the ML pipeline as flawed.
- Decide to redesign the pipeline architecture.
Rubric: The answer must correctly conclude that the overall pipeline is flawed and state that the team needs to redesign it.
0
1
References
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Missing Information in Pipeline Inputs
What is the only valid conclusion when every ML pipeline component is at human-level but the overall pipeline is significantly below human-level?
If every individual component of an ML pipeline achieves human-level performance, the overall pipeline is guaranteed to also achieve human-level performance.
When every component performs at human-level yet the overall pipeline falls short, the pipeline is usually _____ and needs to be redesigned.
Match each pipeline diagnosis concept to its correct description in Ng's framework.
Order the diagnostic steps for determining whether an ML pipeline with human-level components is structurally flawed.
In the self-driving car example, humans given only camera images plan far better paths than the assembled ML pipeline. What does this reveal?
Error analysis on an underperforming ML pipeline can help reveal whether the pipeline structure itself needs to be redesigned.
When assessing a pipeline component against human-level performance, the human baseline must be given the _____ as that component.
Match each pipeline scenario to the correct diagnosis it warrants according to Ng.
Order the reasoning steps Ng uses to conclude that an ML pipeline must be redesigned when all components are already at human-level.
Analyzing Underperforming Pipelines with High-Performing Components
Diagnosing a Self-Driving Car Pipeline
Required Action for Underperforming Pipelines