Question: Describe a situation in which a multi-stage machine learning pipeline is still defective even though each stage performs about as well as a skilled human on its own input. What does that tell you about the pipeline design, and what should be done next?

Sample answer: This happens when each stage looks strong in isolation, but the final system is still much worse than human-level performance overall. The most likely explanation is that important information is not being passed from one stage to the next. In that case, the problem is not just one weak component; the pipeline itself is poorly designed. The fix is to redesign the system so the missing information can flow through the stages.

Key points:

• Each stage performs about as well as a skilled human on its own input.
• The full pipeline still performs much worse than human-level performance.
• The design is missing needed information between stages.
• The pipeline should be redesigned to pass along the missing information.

Rubric: The response must state that strong individual stages do not guarantee a strong end-to-end system, explain that the pipeline is missing information between stages, and conclude that the pipeline design must be changed.