Question: When you estimate how well one stage of a machine learning pipeline performs by comparing it with a person, why should the person be shown only the same inputs that stage receives, rather than the original raw data?

Sample answer: The comparison has to be made with the same information source, otherwise the human is no longer evaluating the same task. For example, if a routing module receives object detections from an earlier vision stage, the human should also work from those detections, not from the full camera feed. Giving the person the full feed lets them use extra clues that the module never sees. Using identical inputs keeps the assessment focused on the stage being tested and avoids blending its errors with mistakes from earlier stages.

Key points:

• The human must not see information that the module does not receive.
• Matching inputs makes the comparison about one component instead of the whole pipeline.
• Upstream mistakes should not be mixed into the score for the downstream stage.

Rubric: Answers should explain that using the original raw data gives the human extra information and makes the comparison unfair. They should also state that the evaluator must receive the same inputs as the pipeline component so that the measurement isolates that component’s performance.