Flawed Pipeline Despite Human-Level Components
If each individual component of an ML pipeline is at or near human-level performance but the overall pipeline falls far short of human-level performance, the pipeline is usually flawed and needs redesign.
0
1
References
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Related
Component Far from Human-Level Performance as Improvement Priority
Flawed Pipeline Despite Human-Level Components
When comparing the Plan path component to human-level performance in a self-driving pipeline, what inputs should the human evaluator receive?
ML Yearning describes comparing pipeline components to human-level performance as a rigorous, procedure-based debugging technique.
In informal pipeline debugging, you ask how _____ each component is from human-level performance.
Match each self-driving pipeline component from ML Yearning to its primary output or function.
Order the informal debugging questions ML Yearning recommends asking when evaluating a self-driving car pipeline.
Why does ML Yearning require the human evaluator for path planning to use only the detection components' outputs rather than camera images?
In ML Yearning's self-driving example, both the Detect cars and Detect pedestrians components feed their outputs directly into the Plan path component.
To compare the Plan path component fairly to a human, the human must plan the route using only the _____ of the detection components.
Match each informal debugging question to the pipeline element it targets in ML Yearning's self-driving car example.
Arrange the reasoning steps a practitioner follows when diagnosing a self-driving pipeline problem using human-level comparison.
Analyzing Pipeline Component Inputs for Human-Level Performance Comparison
Evaluating a Self-Driving Path Planning Component
Input Constraint in Human-Level Comparison for Path Planning
Learn After
Missing Information in Pipeline Inputs
What is the only valid conclusion when every ML pipeline component is at human-level but the overall pipeline is significantly below human-level?
If every individual component of an ML pipeline achieves human-level performance, the overall pipeline is guaranteed to also achieve human-level performance.
When every component performs at human-level yet the overall pipeline falls short, the pipeline is usually _____ and needs to be redesigned.
Match each pipeline diagnosis concept to its correct description in Ng's framework.
Order the diagnostic steps for determining whether an ML pipeline with human-level components is structurally flawed.
In the self-driving car example, humans given only camera images plan far better paths than the assembled ML pipeline. What does this reveal?
Error analysis on an underperforming ML pipeline can help reveal whether the pipeline structure itself needs to be redesigned.
When assessing a pipeline component against human-level performance, the human baseline must be given the _____ as that component.
Match each pipeline scenario to the correct diagnosis it warrants according to Ng.
Order the reasoning steps Ng uses to conclude that an ML pipeline must be redesigned when all components are already at human-level.