Problems Where Humans Perform Poorly Have Weaker Human-Level Baselines
For tasks humans are not good at, such as recommendations, ad selection, or stock prediction, human-level comparison is less useful: labels are harder to obtain, human intuition is harder to rely on, and it is difficult to know the optimal error rate or a reasonable desired error rate.
0
1
References
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Problems Where Humans Perform Poorly Have Weaker Human-Level Baselines
Which characteristic of a task makes it difficult to estimate the optimal error rate, per Machine Learning Yearning?
Predicting what movie to recommend is cited in Machine Learning Yearning as a task where even humans have a hard time.
For tasks that even humans find hard, estimating the _____ error rate is especially difficult.
Match each concept to its correct description regarding optimal error rate estimation.
Order the steps for assessing whether human performance can guide your optimal error rate target.
Which task does Machine Learning Yearning specifically name as an example where human difficulty makes optimal error rate estimation hard?
Human-level performance is always a reliable proxy for the optimal error rate, regardless of whether humans can do the task well.
Alongside movie recommendation, Machine Learning Yearning cites predicting what _____ to show a user as a task even humans find difficult.
Match each scenario to its implication for optimal error rate estimation.
Order the reasoning steps when working on a task where even humans have a hard time.
Consequences of human difficulty on error rate targets
Setting baselines for an ad recommendation engine
Tasks hindering optimal error rate estimation
Learn After
For tasks like book recommendations, ad selection, or stock prediction, why is human-level performance a weaker baseline?
True or False: It is hard for human labelers to annotate a database of users with the 'optimal' book recommendation.
Pretty much no one can predict the _____.
Match each problem with tasks humans perform poorly at to its explanation from the source.
Order the reasoning steps for why human-level comparison is weak for a task like ad selection.
Explain why human-level performance is a weaker baseline for tasks like stock prediction compared to tasks like image labeling.
Diagnose why a book recommendation team can't easily set a target error rate for their system.
Name two examples of tasks the source gives as ones humans are not good at.
Which statement best explains why 'human intuition is harder to count on' for tasks like stock prediction?
True or False: Human-level performance provides an equally useful baseline for all machine learning tasks.