Consequences of human difficulty on error rate targets
Question: Why is it problematic to rely on human performance to set targets when building a system to recommend movies or ads? Discuss the relationship between human capability and the optimal error rate based on the provided text.
Sample answer: When humans have a hard time solving a problem, such as deciding which movie to recommend or ad to show, they do not provide a strong baseline. Because human performance is poor on these tasks, it cannot serve as a reliable proxy, making the optimal error rate very hard to estimate.
Key points:
- Humans struggle with tasks like movie recommendations or ad targeting.
- Poor human performance cannot serve as a reliable baseline.
- This makes the optimal error rate hard to estimate.
Rubric: The essay should identify that poor human performance removes a reliable baseline, making the optimal error rate hard to estimate, and should cite movies or ads as examples.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Problems Where Humans Perform Poorly Have Weaker Human-Level Baselines
Which characteristic of a task makes it difficult to estimate the optimal error rate, per Machine Learning Yearning?
Predicting what movie to recommend is cited in Machine Learning Yearning as a task where even humans have a hard time.
For tasks that even humans find hard, estimating the _____ error rate is especially difficult.
Match each concept to its correct description regarding optimal error rate estimation.
Order the steps for assessing whether human performance can guide your optimal error rate target.
Which task does Machine Learning Yearning specifically name as an example where human difficulty makes optimal error rate estimation hard?
Human-level performance is always a reliable proxy for the optimal error rate, regardless of whether humans can do the task well.
Alongside movie recommendation, Machine Learning Yearning cites predicting what _____ to show a user as a task even humans find difficult.
Match each scenario to its implication for optimal error rate estimation.
Order the reasoning steps when working on a task where even humans have a hard time.
Consequences of human difficulty on error rate targets
Setting baselines for an ad recommendation engine
Tasks hindering optimal error rate estimation