Google

A pipeline can be poorly designed if the full system performs much worse than human level even though each stage is close to human level on the information it receives. In that case, the weakness is often that important information is being lost, so the design should be reconsidered.

When a Pipeline Is Missing Information

One way to build a speech recognizer is to first extract MFCC features from audio, then predict a sequence of phonemes, and finally convert those phonemes into the written transcript.

A Multi-Stage Speech Recognition System

In a modular self-driving system, car detection, pedestrian detection, and path planning can each perform well on their own inputs, yet the full system may still do poorly on camera scenes. That usually means the pipeline is missing a useful cue. Adding a lane-line detector can supply that missing road-context information so the planner does not need to interpret raw camera images directly.

When a Driving Pipeline Lacks the Right Information

A manually designed processing stage can reduce final performance if it removes useful detail or converts the input into an imperfect proxy. For example, in an audio system, MFCCs compress the signal and phoneme labels summarize speech with a coarse, human-made representation rather than the full waveform.

Handcrafted Stages Can Cap Overall System Quality

When labeled data is limited, carefully designed features or structures can make the learning task easier and reduce the sample size needed to get useful performance. In speech tasks, mel-frequency cepstral coefficients can suppress details that are less important, such as speaker-specific pitch or room effects, while phoneme-like categories give the model a more compact way to represent recurring sound units.

Domain-Designed Features Can Lower the Amount of Training Data Needed

If you decide not to build an end-to-end system, you still have to make two design choices: which stages the pipeline should include and how those stages should pass information to one another.

Choosing Steps and Connections in a Pipeline

When designing a multi-stage system instead of an end-to-end model, a key criterion is whether enough training examples can be collected for each stage.

Choose Pipeline Stages You Can Train

Learned and Non-Learned Pipeline Components

If a hospital intake system has highly accurate symptom and risk classifiers, its end-to-end patient routing must also be highly accurate.

A multi-stage system works only if you can _____ enough labeled examples for each component.

Three Ideas in a Machine Learning Pipeline

How to Rework a Multi-Stage Machine Learning Pipeline

Question:
Describe a situation in which a multi-stage machine learning pipeline is still defective even though each stage performs about as well as a skilled human on its own input. What does that tell you about the pipeline design, and what should be done next?

Sample answer:
This happens when each stage looks strong in isolation, but the final system is still much worse than human-level performance overall. The most likely explanation is that important information is not being passed from one stage to the next. In that case, the problem is not just one weak component; the pipeline itself is poorly designed. The fix is to redesign the system so the missing information can flow through the stages.

Key points:
- Each stage performs about as well as a skilled human on its own input.
- The full pipeline still performs much worse than human-level performance.
- The design is missing needed information between stages.
- The pipeline should be redesigned to pass along the missing information.

Rubric:
The response must state that strong individual stages do not guarantee a strong end-to-end system, explain that the pipeline is missing information between stages, and conclude that the pipeline design must be changed.

When a Multi-Stage Pipeline Hides Missing Information

Case context:
You are building a parcel-delivery drone system. The pipeline has two main parts: an image-based landing-zone detector and a rule-based route planner. When each part is tested separately on the inputs it receives, both appear to work at about human level. But when the full system is run end to end, the drone often chooses unsafe approaches and misses suitable landing spots.

Question:
Using the performance of the separate parts and the complete system, diagnose what is wrong with the pipeline and decide what action to take next.

Sample answer:
The most likely problem is that the pipeline design is flawed because important information is not passing cleanly from one stage to the next. The detector may be doing its job, but the planner is not getting enough useful detail to make good decisions. The next step is to redesign the pipeline so the later stage receives the information it needs, which may require changing the intermediate representation or making the stages work more closely together.

Key points:
- Diagnose the issue as a pipeline or interface problem.
- Recognize that missing information is causing the poor end-to-end result.
- Recommend redesigning the information flow between stages.

Rubric:
The response must identify the pipeline as flawed due to missing information and conclude that the pipeline should be redesigned.

Case Study: Diagnosing a Parcel-Delivery Drone Pipeline

Question:
Name one component in a robot system pipeline that can be implemented without machine learning.

Sample answer:
A conventional route-planning algorithm such as A* or Dijkstra can be used for the final path-planning stage without learning from data.

Key points:
- Classical route-planning or path-planning algorithm.
- Used in the final planning stage.
- Does not require training on data.

Rubric:
The answer should identify a non-learned planning component, such as a classical path-planning algorithm, as an example of a pipeline stage that does not involve learning.

Learn Before

Related