Diagnosing a self-driving car pipeline.
Case context: A self-driving car uses a pipeline with three steps: Car Detection (A), Pedestrian Detection (B), and Path Planning (C). On a specific dev-set example, the car makes a poor path planning decision. You provide a "perfect" bounding box for Car Detection (A), but the final path is still incorrect. You then provide a "perfect" output for Pedestrian Detection (B), and the final path becomes correct.
Question: Based on the general error attribution procedure, what should the engineer diagnose as the cause of the poor path planning decision for this specific dev-set mistake, and why?
Sample answer: The engineer should diagnose Pedestrian Detection (B) as the cause of the error. The procedure dictates that if replacing A's output with a perfect one does not fix the system, you move to B. Since providing a perfect output for B resulted in a correct final path, it shows that if B had produced a better output, the overall system would have been correct. Therefore, the error is attributed to B.
Key points:
- Component B (Pedestrian Detection) is the source of the error.
- Providing perfect input to A did not fix the problem.
- Providing perfect input to B fixed the overall system output.
Rubric: The response must identify component B (Pedestrian Detection) as the source of the error and correctly justify it using the logic of the error attribution procedure.
0
1
References
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Self-Driving Car Error Attribution Scenario
DAG Ordering for Pipeline Error Attribution
What is the purpose of manually replacing a component's output with a 'perfect' output in the error attribution procedure?
If replacing A's output with a perfect output causes the pipeline to produce a correct result, the error should be attributed to component B.
In a pipeline A → B → C, if replacing A's output with a perfect output fixes the system, you attribute the error to _____.
Match each error attribution scenario to the correct conclusion about which pipeline component caused the error.
Order the steps of the general error attribution procedure for a three-step pipeline A → B → C.
In a pipeline A → B → C, under what condition does the error attribution procedure conclude that component C caused the error?
The error attribution procedure requires retraining every pipeline component from scratch for each dev-set mistake analyzed.
The error attribution procedure is applied to each _____ the system makes on the dev set, one at a time.
Match each pipeline component in A → B → C to the diagnostic action taken when testing whether it is the source of a dev-set error.
Order the reasoning steps for deciding which component to prioritize for improvement after running error attribution across many dev-set mistakes.
Describe the error attribution process for a three-step pipeline.
Diagnosing a self-driving car pipeline.
Default component for pipeline errors.