Learn Before
Perfect-Input Check for Locating a Pipeline Error
To identify which stage in a multi-step system is causing a mistake, replace the output from an earlier stage with a manually verified correct output and then run the later stage on that input. If the later stage still produces the error, the later stage is likely responsible. If the error disappears, the earlier stage is the likely source.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Perfect-Input Check for Locating a Pipeline Error
Why is it hard to assign blame when a pipeline's first step creates a poor intermediate result?
In a package-tracking system, only a few cases are hard to tell apart between a barcode reader error and a route-classification error. Does it matter much which one you assign those rare cases to when you summarize the main error pattern?
When ambiguous attribution cases are scarce, the exact assignment of those cases will not materially change the overall error analysis.
When is blame assignment in a multi-stage machine learning system hardest to pin down?
A person with good judgment could still identify a chair from a badly framed crop made by a weak detector.
Crop Quality Problem in an Image Pipeline
Match each pipeline attribution concept to its correct description.
Order the steps for deciding whether attribution ambiguity is worth worrying about.
When a Two-Stage Detector and Classifier Disagree
In a customer-support review, if only a few tickets could plausibly belong to more than one category, the exact category you assign those tickets will usually change the overall statistics a lot.
If the number of unclear pipeline mistakes is _____, you can assign them either way and the overall conclusion will change very little.
Match each part of a two-stage license-plate system to its role in diagnosing the error.
Put the steps of the document-processing error example in the correct order, from the first module to the final attribution question.
Why can error attribution be unclear in a multi-stage workflow?
Attributing errors in a berry detector and ripeness classifier pipeline
When is any error assignment acceptable for a small set of ambiguous pipeline cases?
Learn After
Using Component Errors to Focus Improvement
What should replace the first stage output in a perfect-input check for a bird detector and species classifier pipeline?
True or False: If a crop classifier still mislabels an image even after the fruit region has been perfectly marked and isolated, the crop-detection stage is the cause of the remaining error.
In the perfect input test, if the traffic-sign classifier now correctly outputs y=1 when given a hand-labeled box, the original mistake is attributed to the _____.
Match each outcome of a perfect-input check to the component that should be held responsible for the pipeline error.
Order the steps in the perfect-input check for isolating an error between a traffic sign detector and a sign-type classifier.
How a perfect intermediate result pinpoints where a two-stage system fails
Apply a perfect-input check to isolate the source of a two-stage vision failure.
What must an engineer prepare to run the perfect input test, and why must it be exact?
Attributing Error After a Perfect Input Check
True or False: The perfect input test can help isolate which stage in a pipeline is responsible for an error.