Choose a pipeline structure when intermediate labels are abundant
Case context: You are building a navigation system for an indoor delivery robot. You have 250,000 labeled images showing the locations of boxes, carts, and other hallway obstacles, but only 14,000 examples that map the robot’s raw sensor readings directly to steering commands.
Question: Given the data you have for intermediate steps in the task, what pipeline structure should you choose, and why?
Sample answer: I would choose a multi-stage pipeline. The large obstacle-location dataset is ideal for training an intermediate detector that finds objects in the hallway. Since there is much more data for that intermediate module than for the full sensor-to-command task, a staged design makes better use of the available labels than trying to force the whole problem into a single end-to-end model.
Key points:
- The obstacle-location labels are useful for an intermediate detection module
- A multi-stage pipeline fits a situation with plentiful intermediate data
- The detector can be trained directly on the large labeled set
Rubric: The learner must identify the obstacle-location data as training data for an intermediate module and conclude that a multi-stage pipeline is the better choice because it can exploit that larger dataset.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Why Labeled Perception Data Can Make a Staged Driving System Easier
When is a staged model design especially attractive?
A staged design can be better when each stage has usable training data.
Available data for _____ in a staged workflow
Match staged system concepts
Choosing a Multi-Stage Pipeline When Subtask Labels Are Abundant
When a Modular Pipeline Can Be Competitive
Choose a pipeline structure when intermediate labels are abundant
When can a staged pipeline outperform a single direct model?
Which example is an intermediate module in a larger machine learning pipeline?
Limited data makes a multi-stage pipeline the best choice.