Google

For a self-driving car pipeline, a closed-track scenario where the car chooses a more jarring steering direction than a skilled driver can be analyzed by manually perfecting the detect-cars output, then the detect-pedestrians output, and finally attributing remaining errors to path planning.

Self-Driving Car Error Attribution Scenario

Machine Learning Yearning, a free ebook from Andrew Ng, teaches you how to structure Machine Learning projects.
https://www.deeplearning.ai/machine-learning-yearning/

Note: The content of the book is aligned with the Coursera Deeplearning.ai specialization.  https://www.deeplearning.ai/deep-learning-specialization/ 

Machine Learning Yearning (Deeplearning.ai)

In the self-driving car jarring-steering scenario, which pipeline component is manually perfected first when attributing the error?

True or False: If perfecting component A's output makes the pipeline plan a much better path, the error is attributed to A.

In the self-driving world, a closed-track case where the car steers more jarringly than a skilled driver would is called a _____.

Match each self-driving car pipeline component to its function.

Order the steps for attributing the jarring-steering error to component A in the self-driving car pipeline.

Question:
Using the self-driving car scenario as an example, explain why manually substituting a 'perfect' output for one component (such as detect cars) and re-running the rest of the pipeline helps determine whether that component is responsible for a pipeline error.

Sample answer:
By manually perfecting component A's output (e.g., manually specifying where other cars are) and letting the downstream components B and C run as before, you create a controlled comparison: the only thing that changed is A's output quality. If the final path planning output improves substantially when A is perfect, this proves that A's original, imperfect output was the bottleneck causing the poor path decision, because everything else in the pipeline was held constant. If the output does not improve, A's errors are not the main cause, and the fault likely lies with a downstream component like B or C. This isolates the source of error without needing to retrain or redesign the whole pipeline.

Key points:
- Only one component's output is changed at a time
- Downstream components run as before to keep the comparison controlled
- A large improvement in the final output implicates the perfected component
- No improvement means the fault lies with a downstream component
- This avoids retraining the whole pipeline to find the error source

Rubric:
Full credit explains that holding other components constant while perfecting one isolates its causal contribution to the final error; also credits noting that no improvement rules out A and points to later components.

Why does manually perfecting one component's output help isolate which component caused a pipeline error?

Case context:
An engineer tests a self-driving car on a closed track and finds a scenario where the car chooses a noticeably more jarring steering direction than a skilled human driver would. The pipeline consists of detect-cars (A), detect-pedestrians (B), and plan-path (C).

Question:
Describe the specific steps the engineer should take to determine whether component A (detect cars) is responsible for this jarring steering scenario, and how they should interpret the result.

Sample answer:
The engineer should manually modify A's output for this scenario to be 'perfect,' for example by manually specifying the true positions of the other cars. They should then re-run B (detect pedestrians) as normal and C (plan path), but allow C to use A's newly perfected output. If the pipeline now plans a much smoother, less jarring path, this shows that A's original detection error was responsible for the bad path decision, so the error should be attributed to component A. If the path is still jarring even with a perfect A, component A is not the cause, and the engineer should move on to test component B next using the same approach.

Key points:
- Manually perfect component A's output for the scenario
- Re-run B normally and C using A's perfected output
- A much better path indicates the error is attributable to A
- No improvement means A is not the cause and B should be tested next
- The comparison isolates A's contribution to the overall error

Rubric:
Full credit describes manually perfecting A's output, re-running B and C with C using the perfected A output, and correctly interprets both a positive and negative result; partial credit for describing only part of the procedure.

Diagnose the source of a jarring steering decision using error attribution by parts.

Question:
In the self-driving car error attribution example, what does it mean, and what should you do next, if manually perfecting component A's (detect cars) output does not improve the path planning output?

Sample answer:
It means component A is not the source of the error in this scenario, since even with a perfect A output the path is still jarring. The engineer should then move on to Step 2 of the procedure and manually perfect component B's (detect pedestrians) output to test whether B is responsible.

Key points:
- No improvement rules out component A as the cause
- The procedure moves on to testing component B next
- This keeps the analysis systematic across pipeline components

Rubric:
Full credit states that A is ruled out as the cause and that the next step is to test component B; partial credit for only stating one of these.

Learn Before

Related