Learn Before
Fair Human Comparison for a Route-Planning Module
Question: When comparing a vehicle's route-planning module to human-level performance, what limitation should be placed on the human's inputs so the comparison is fair?
Sample answer: The human should make the routing decision using only the outputs of the earlier pipeline stages, such as detected obstacles or lane information, rather than the raw sensor feed.
Key points:
- The human may only see the upstream system outputs that the planner receives.
- The human must not use the original raw sensor data.
Rubric: The answer must state that the human is restricted to the outputs from preceding components and cannot use the raw sensor input.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Far-Behind Components Are Good Improvement Targets
Strong Stages, Weak End-to-End System
What information should a human evaluator see when judging the Route Planning component in a delivery-robot pipeline?
Checking a subsystem against human performance is mainly a formal, rigid debugging procedure.
How far is a component from human performance?
Match each autonomous delivery robot component to its primary output or function.
Order the informal debugging questions for a package-delivery drone system.
Why should the human reviewer for a parcel-routing module use only the classifier's outputs instead of the warehouse camera feed?
In a warehouse robot pipeline, both the shelf-item detector and the obstacle detector can send information directly to the route planner.
A fair human comparison uses the same inputs
Match each diagnostic question to the part of a medical triage pipeline it evaluates.
Diagnosing a Multi-Stage Perception Pipeline
Why Human Comparisons Must Use the Same Component Inputs
Comparing a Delivery Robot Planner to Human Performance
Fair Human Comparison for a Route-Planning Module