Learn Before
Troubleshooting an Autonomous Delivery System
Case context: Your team is building an autonomous delivery cart that navigates sidewalks and campus paths. When you test the subsystems separately, the pedestrian detector, obstacle tracker, and route chooser each match expert-level performance. But when the full cart is run end to end on the same sensor data, its route choices are much worse than those made by a skilled human operator.
Question: Using Andrew Ng's framework, what is the core issue in this system, and what action should the team take?
Sample answer: The core issue is that the overall machine learning pipeline is not working properly. Even though the individual modules perform at human level, the integrated system does not, so the architecture needs to be redesigned. The team should rework the pipeline rather than assume the problem is only one module.
Key points:
- Identify the full pipeline as the source of the problem.
- Choose to redesign the pipeline architecture.
Rubric: The answer must conclude that the end-to-end pipeline is flawed and that the team needs to redesign it.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
When a Processing Stage Lacks the Right Input
If every component of a machine learning system matches human performance, but the full system is still much worse than human performance, what should you conclude?
Perfect Subsystems Always Produce a Perfect Pipeline
If every stage of a learning system looks strong on its own but the full system still underperforms, the overall design is usually _____ and should be rethought.
Match each pipeline diagnosis idea to its meaning.
Order the diagnostic steps for checking whether a multi-stage model is poorly designed even when each stage seems strong on its own.
In a warehouse-robot example, people who see only the robot's camera feeds choose better routes than the full machine-learning pipeline, even though each module has been tuned to human level. What is the best conclusion?
Error analysis on a weak machine learning pipeline can reveal that the overall system design should be changed.
When comparing a system module to human performance, the human should receive the _____ that the module receives.
Match each system outcome to the most reasonable diagnosis.
Put these steps in the order that shows why a model pipeline should be redesigned when every component is already performing at human level.
What a Strong Pipeline Can Still Get Wrong
Troubleshooting an Autonomous Delivery System
What to Do When an End-to-End System Is Still Weak